-- Logs begin at Thu 2022-06-23 00:49:11 UTC, end at Thu 2022-06-23 02:14:25 UTC. -- Jun 23 01:42:32.096231 dv-cinder-815582-9-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 23 01:42:32.105664 dv-cinder-815582-9-iscsi devstack@keystone.service[80448]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 23 01:42:32.107628 dv-cinder-815582-9-iscsi devstack@keystone.service[80448]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jun 23 01:42:32 2022] *** Jun 23 01:42:32.107628 dv-cinder-815582-9-iscsi devstack@keystone.service[80448]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 23 01:42:32.107628 dv-cinder-815582-9-iscsi devstack@keystone.service[80448]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 23 01:42:32.107628 dv-cinder-815582-9-iscsi devstack@keystone.service[80448]: nodename: dv-cinder-815582-9-iscsi Jun 23 01:42:32.107628 dv-cinder-815582-9-iscsi devstack@keystone.service[80448]: machine: x86_64 Jun 23 01:42:32.107628 dv-cinder-815582-9-iscsi devstack@keystone.service[80448]: clock source: unix Jun 23 01:42:32.107628 dv-cinder-815582-9-iscsi devstack@keystone.service[80448]: pcre jit disabled Jun 23 01:42:32.107628 dv-cinder-815582-9-iscsi devstack@keystone.service[80448]: detected number of CPU cores: 4 Jun 23 01:42:32.107628 dv-cinder-815582-9-iscsi devstack@keystone.service[80448]: current working directory: / Jun 23 01:42:32.107628 dv-cinder-815582-9-iscsi devstack@keystone.service[80448]: detected binary path: /usr/bin/uwsgi-core Jun 23 01:42:32.108635 dv-cinder-815582-9-iscsi devstack@keystone.service[80448]: your processes number limit is 47680 Jun 23 01:42:32.108635 dv-cinder-815582-9-iscsi devstack@keystone.service[80448]: your memory page size is 4096 bytes Jun 23 01:42:32.108635 dv-cinder-815582-9-iscsi devstack@keystone.service[80448]: detected max file descriptor number: 2048 Jun 23 01:42:32.108635 dv-cinder-815582-9-iscsi devstack@keystone.service[80448]: lock engine: pthread robust mutexes Jun 23 01:42:32.108635 dv-cinder-815582-9-iscsi devstack@keystone.service[80448]: thunder lock: enabled Jun 23 01:42:32.108635 dv-cinder-815582-9-iscsi devstack@keystone.service[80448]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 23 01:42:32.108635 dv-cinder-815582-9-iscsi devstack@keystone.service[80448]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 23 01:42:32.152671 dv-cinder-815582-9-iscsi devstack@keystone.service[80448]: Python main interpreter initialized at 0x55fb4d1059c0 Jun 23 01:42:32.152671 dv-cinder-815582-9-iscsi devstack@keystone.service[80448]: python threads support enabled Jun 23 01:42:32.152671 dv-cinder-815582-9-iscsi devstack@keystone.service[80448]: your server socket listen backlog is limited to 100 connections Jun 23 01:42:32.152671 dv-cinder-815582-9-iscsi devstack@keystone.service[80448]: your mercy for graceful operations on workers is 90 seconds Jun 23 01:42:32.153053 dv-cinder-815582-9-iscsi devstack@keystone.service[80448]: mapped 403077 bytes (393 KB) for 2 cores Jun 23 01:42:32.153053 dv-cinder-815582-9-iscsi devstack@keystone.service[80448]: *** Operational MODE: preforking *** Jun 23 01:42:32.153359 dv-cinder-815582-9-iscsi devstack@keystone.service[80448]: *** uWSGI is running in multiple interpreter mode *** Jun 23 01:42:32.153359 dv-cinder-815582-9-iscsi devstack@keystone.service[80448]: spawned uWSGI master process (pid: 80448) Jun 23 01:42:32.153545 dv-cinder-815582-9-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 23 01:42:32.155247 dv-cinder-815582-9-iscsi devstack@keystone.service[80448]: spawned uWSGI worker 1 (pid: 80459, cores: 1) Jun 23 01:42:32.155247 dv-cinder-815582-9-iscsi devstack@keystone.service[80448]: spawned uWSGI worker 2 (pid: 80460, cores: 1) Jun 23 01:42:32.155247 dv-cinder-815582-9-iscsi devstack@keystone.service[80448]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 23 01:42:37.501747 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 23 01:42:37.501747 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: from cryptography.utils import int_from_bytes Jun 23 01:42:37.502566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 23 01:42:37.502566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: from cryptography.utils import int_from_bytes Jun 23 01:42:37.502566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 23 01:42:37.502566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: from cryptography.utils import int_from_bytes Jun 23 01:42:37.503063 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 23 01:42:37.503063 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: from cryptography.utils import int_from_bytes Jun 23 01:42:37.550655 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 23 01:42:37.550822 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 23 01:42:37.550900 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 23 01:42:37.550976 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 23 01:42:37.551042 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] command line args: [] {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 23 01:42:37.551106 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] command line args: [] {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 23 01:42:37.551174 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 23 01:42:37.551265 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 23 01:42:37.551406 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ================================================================================ {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 23 01:42:37.551532 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ================================================================================ {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 23 01:42:37.551608 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] admin_token = **** {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.551673 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] admin_token = **** {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.551855 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] config_dir = [] {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.551951 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] config_dir = [] {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.552017 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.552092 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.552209 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] config_source = [] {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.552298 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] config_source = [] {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.552370 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] debug = True {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.552445 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] debug = True {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.552588 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.552834 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.552951 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.553013 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.553070 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.553126 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.554036 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.554146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.554146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.554268 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.554337 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] list_limit = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.554395 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] list_limit = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.554456 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] log_config_append = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.554524 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] log_config_append = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.554582 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.554645 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.554707 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] log_dir = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.554763 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] log_dir = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.554825 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] log_file = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.554891 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] log_file = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.554954 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.555016 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.555079 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.555142 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.555199 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.555320 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.555387 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.555447 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.555797 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.556294 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.556388 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.556472 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.556555 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.556627 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.556700 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.556770 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.556847 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.556917 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.556992 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.557064 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.557210 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.557301 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.557419 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.557534 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.557754 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.557879 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.557961 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.558021 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.558181 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.558313 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.558662 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.558750 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.558822 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] publish_errors = False {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.558881 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] publish_errors = False {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.559050 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.559168 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.559301 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.559381 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.559494 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.559567 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.559710 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.559810 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.559941 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.560019 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.560174 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] standard_threads = False {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.560253 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] standard_threads = False {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.560401 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.560546 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.560627 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.560699 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.560805 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.560883 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.560965 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] use_journal = False {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.561039 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] use_journal = False {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.561135 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] use_json = False {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.561212 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] use_json = False {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.561333 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] use_stderr = False {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.561409 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] use_stderr = False {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.561488 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] use_syslog = False {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.561566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] use_syslog = False {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.561682 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.561759 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 01:42:37.561932 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.562008 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.562167 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.562243 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.562418 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.562502 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.562654 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.562747 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.562897 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.562975 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.563132 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.563233 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.563410 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.563502 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.563626 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.563711 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.563871 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.563953 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.564111 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.564192 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.564358 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.564435 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.564586 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.564667 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.564827 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.564903 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.565058 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.565135 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.565300 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.565383 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.565544 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.565620 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.565774 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.565853 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.566009 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.566093 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.566259 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.566335 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.566497 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.566579 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.566729 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.566811 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.566976 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.567058 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.567250 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.567332 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.567504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.567582 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.567747 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.567824 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.567974 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.568051 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.568200 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.568282 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.568427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.568514 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.568655 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.568731 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.568920 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.569107 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.569201 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.569287 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.569428 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.569504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.569654 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.569735 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.569895 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.569981 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.570138 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.570214 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.570370 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.570455 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.570612 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.570697 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.570840 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.570915 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.571068 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.571154 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.571319 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.571479 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.571555 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.571626 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.571763 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.571846 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.571988 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.572073 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.572214 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.572290 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.572446 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.572522 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.572687 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.572771 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.572915 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.572991 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.573146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.573228 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.573370 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.573454 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.573601 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.573678 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.573835 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.573910 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.574079 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.574192 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.574308 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.574407 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.574539 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.574639 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.574764 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.574871 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.574988 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.575112 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.575248 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.575381 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.575459 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.575584 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.575698 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.575773 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.575912 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.575994 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.576144 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.576223 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.576382 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.576460 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.576608 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.576689 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.576861 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.576947 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.577094 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.577181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.577321 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.577399 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.577554 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.577632 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.577785 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.577874 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.578046 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.578124 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.578283 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.578359 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.578524 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.578608 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.578763 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.578890 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.578996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.579079 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.579241 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.579323 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.579478 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.579558 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.579723 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.579802 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.579956 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.580033 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.580188 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.580266 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.580422 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.580498 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.580653 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.580730 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.580876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.580954 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.581107 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.581189 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.581335 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.581412 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.581568 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.581644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.581823 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.581910 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.582057 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.582136 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.582294 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.582375 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.582526 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.582611 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.582760 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.582834 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.582985 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.583064 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.583225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.583303 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.583462 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.583550 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.583686 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.583765 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.583911 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.583988 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.584135 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.584212 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.584365 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.584444 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.584615 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.584692 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.584850 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.584962 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.585069 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.585147 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.585311 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.585387 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.585543 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.585620 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.585789 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.585870 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.586042 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.586125 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.586278 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.586367 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.586505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] auth.external = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.586583 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] auth.external = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.586730 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.586814 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.587020 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.587126 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.587271 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.587368 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.587500 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] auth.password = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.587577 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] auth.password = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.587726 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] auth.token = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.587803 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] auth.token = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.587957 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.588041 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.588187 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.588265 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.588426 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.588502 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.588653 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.588743 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.588898 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.588976 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.589140 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.589215 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.589368 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.589458 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.589600 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] credential.caching = True {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.589680 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] credential.caching = True {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.589841 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.589919 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.590085 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.590170 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.590325 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.590402 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.590557 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.590635 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.590795 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.590881 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.591025 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.591103 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.591301 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.591384 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.591551 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.591640 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.591780 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.591858 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.592029 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.592108 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.592305 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.592390 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.592547 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.592625 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.592806 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.592892 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.593043 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.593128 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.593295 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.593372 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.593526 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] federation.caching = True {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.593603 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] federation.caching = True {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.593758 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.593843 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.593999 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.594077 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.594237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.594314 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.594464 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.594556 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.594704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.594782 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.594941 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.595027 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.595188 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.595292 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.595443 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.595529 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.595692 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.595770 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.595921 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.596007 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.596162 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.596239 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.596390 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] identity.caching = True {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.596473 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] identity.caching = True {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.596634 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.596790 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.596865 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.596936 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.597091 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.597170 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.597322 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.597402 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.597563 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.597641 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.597787 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.597872 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.598019 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.598099 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.598262 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.598340 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.598504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.598581 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.598729 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.598809 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.598954 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.599032 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.599180 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.599288 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.599440 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.599523 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.599680 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.599758 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.599920 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.600006 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.600216 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.600295 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.600465 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.600544 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.600732 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.600808 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.600965 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.601044 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.601197 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.601274 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.601427 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.601502 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.601660 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.601738 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.601885 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.601961 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.602115 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.602193 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.602352 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.602430 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.602584 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.602669 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.602818 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.602896 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.603036 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.603116 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.603299 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.603384 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.603542 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.603620 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.603772 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.603851 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.604010 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.604086 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.604248 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.604325 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.604477 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.604557 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.604709 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.604786 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.604933 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.605012 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.605166 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.605244 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.605399 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.605483 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.605634 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.605711 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.605868 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.605945 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.606096 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.606172 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.606338 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.606416 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.606580 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.606659 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.606806 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.606889 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.607032 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.607110 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.607301 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.607383 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.607544 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.607622 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.607774 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.607851 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.608004 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.608084 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.608236 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.608314 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.608459 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.user = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.608535 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.user = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.608696 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.608776 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.608949 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.609026 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.609176 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.609261 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.609413 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.609490 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.609653 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.609730 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.609889 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.609974 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.610119 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.610196 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.610339 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.610416 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.610568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.610656 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.610799 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.610876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.611026 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.611103 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.611268 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.611363 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.611514 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.611591 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.611755 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.611832 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.611989 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.612074 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.612228 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.612306 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.612468 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.612547 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.612692 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.612779 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.612939 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.613017 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.613164 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.613242 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.613396 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.613484 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.613626 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.613703 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.613856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.613934 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.614096 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.614182 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.614337 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.614415 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.614567 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.614644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.614812 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.614898 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.615039 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.615115 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.615298 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.615388 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.615534 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.615620 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.615788 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.615866 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.616026 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.616104 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.616266 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.616351 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.616503 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.616580 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.616729 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.616805 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.616961 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.617049 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.617195 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] resource.caching = True {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.617272 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] resource.caching = True {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.617436 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.617513 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.617679 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.617763 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.617913 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.617990 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.618159 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.618237 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.618405 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.618491 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.618636 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.618714 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.618876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.618952 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.619108 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.619302 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.619381 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.619453 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.619582 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] role.caching = True {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.619660 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] role.caching = True {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.619823 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] role.driver = sql {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.619909 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] role.driver = sql {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.620050 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.620128 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.620294 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.620373 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.620536 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.620622 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.620778 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.620856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.621017 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.621093 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.621258 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.621337 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.621501 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.621579 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.621740 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.621816 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.621979 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.622059 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.622207 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.622283 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.622445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.622521 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.622686 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.622766 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.622927 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.623004 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.623166 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.623290 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.623574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.623656 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.623806 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.623883 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.624046 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.624123 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.624291 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.624370 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.624529 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.624606 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.624771 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.624849 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.625000 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.625080 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.625248 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.625326 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.625491 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.625567 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.625724 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.625803 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.625950 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.626027 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.626173 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.626249 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.626399 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.626477 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.626640 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.626717 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.626890 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.626974 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.627131 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.627223 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.627386 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.627466 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.627617 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.627711 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.627843 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.627922 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.628074 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] token.caching = True {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.628151 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] token.caching = True {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.628303 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.628387 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.628544 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.628621 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.628774 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.628850 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.629023 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.629108 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.629261 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.629337 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.629493 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.629569 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.629729 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.629814 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.629964 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.630041 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.630209 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.630286 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.630442 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.630526 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.630674 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.630751 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.630904 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.630981 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.631143 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.631256 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.631457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.631571 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.631693 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.631803 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.631936 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.632045 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.632228 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.632330 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.632499 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] database.connection = **** {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.632586 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] database.connection = **** {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.632743 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.632831 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.632989 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.633067 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.633224 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.633304 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.633467 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.633545 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.633702 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.633780 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.633930 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.634008 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.634159 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.634237 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.634388 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.634466 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.634625 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.634704 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.634854 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.634979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.635089 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.635168 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.635341 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.635423 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.635582 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.635660 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.635812 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.635890 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.636043 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.636140 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.636268 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.636352 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.636501 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.636578 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.636729 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.636807 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 01:42:37.636879 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 23 01:42:37.636951 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80460) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 23 01:42:37.666852 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:37.666998 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:37.667083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:37.667146 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:37.667302 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:37.667387 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:37.675687 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:37.675909 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:37.703541 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:37.703739 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:37.703868 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:37.704049 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:37.845481 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:37.845672 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:37.893086 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:37.893222 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:37.893291 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:37.893376 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:37.956756 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:37.956922 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:37.957006 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:37.957178 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:37.957336 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:37.957336 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:37.957519 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:37.957602 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:37.957664 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:37.957769 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:37.957867 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:37.958025 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:37.982415 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:37.982629 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:37.982742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:37.982892 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:37.982980 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:37.983103 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:37.983185 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:37.983402 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:37.983538 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:37.983667 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:37.983788 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:37.983921 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:37.984043 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:37.984163 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:37.994716 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:37.994988 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:38.007251 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:38.007479 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:38.007693 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:38.007840 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:38.007933 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:38.007999 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:38.008097 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:38.008171 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:38.008339 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:38.008465 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:38.008534 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80460) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:38.008610 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:38.008728 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:38.008974 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 01:42:38.025648 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.025837 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 23 01:42:38.026158 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.026282 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.026350 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 23 01:42:38.026434 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 23 01:42:38.026623 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.026750 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.026822 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 23 01:42:38.026899 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.027106 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.027254 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.027354 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.027443 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.027641 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.027870 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.038077 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.038333 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.038598 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.038721 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.038843 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.038967 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.039194 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.039500 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.047083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 01:42:38.047780 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 01:42:38.050336 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 01:42:38.050608 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.050880 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.051002 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 01:42:38.051153 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.051307 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.051412 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.051578 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.051703 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.051837 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.051959 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.052139 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.052262 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.052405 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.052528 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.052634 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.052723 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.052899 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.052989 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.053181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.053502 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.053747 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.079345 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.080250 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.080926 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 01:42:38.081197 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.081845 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 01:42:38.082116 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.086491 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 01:42:38.086875 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.087147 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.087416 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 01:42:38.087811 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.088089 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.094355 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 01:42:38.094630 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.095337 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 01:42:38.095662 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.099053 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 01:42:38.099357 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.099631 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.099905 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.100029 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 01:42:38.100192 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.100328 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.100467 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.100616 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.100708 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.100837 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.100920 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.101137 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.101406 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.101668 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.101930 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.119476 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.119751 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.120591 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.120866 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.124338 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 01:42:38.125495 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 01:42:38.127949 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.128339 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.129129 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.129522 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.133361 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 01:42:38.134535 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 01:42:38.136909 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 01:42:38.138159 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 01:42:38.140565 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.140861 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.141139 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.141418 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.141688 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.141810 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.141965 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.142090 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.142350 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.142613 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.142876 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.143180 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.160892 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 01:42:38.161176 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.161439 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.161702 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.162112 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 01:42:38.162393 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.162679 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.162950 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.169551 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.170797 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.171244 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.171520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.172517 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.172790 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.174681 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 01:42:38.174971 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.175374 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.175655 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.175930 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.176055 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 01:42:38.176342 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.176740 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.177056 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.177337 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.189388 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 01:42:38.189665 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.189941 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.190210 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.190827 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 01:42:38.191103 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.191415 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.191694 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.191818 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.191987 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.192260 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.193199 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.193491 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.193784 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.209941 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 01:42:38.211654 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 01:42:38.213210 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 01:42:38.214936 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 01:42:38.216477 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.218182 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.218306 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.219815 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 01:42:38.219961 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.220100 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.220350 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.221568 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 01:42:38.221852 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.222142 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.227635 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.229230 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 01:42:38.229497 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.231091 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 01:42:38.232450 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.232732 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.233110 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.233377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.234323 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.234609 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.234997 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.235288 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.242415 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 01:42:38.242692 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.242962 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.244387 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 01:42:38.244676 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.244950 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.250705 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80460) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 01:42:38.250975 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.251243 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.251542 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.251805 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.252187 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.252455 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.252722 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.252860 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 01:42:38.252933 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.253009 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.253209 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.253334 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.253471 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.253565 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.253750 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.253877 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.253974 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.254182 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.254310 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80460) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.254463 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.254711 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.254986 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.255281 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.255551 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.255828 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.256092 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.256382 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 01:42:38.291626 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55fb4d1059c0 pid: 80460 (default app) Jun 23 01:42:38.294208 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55fb4d1059c0 pid: 80459 (default app) Jun 23 01:42:38.295040 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482d2ac-5114-4e76-8b5f-440c708aa7ca None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:38.295372 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482d2ac-5114-4e76-8b5f-440c708aa7ca None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:38.295644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482d2ac-5114-4e76-8b5f-440c708aa7ca None None] PATH_INFO: `/v3` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:38.299829 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1/1] 10.222.0.26 () {50 vars in 791 bytes} [Thu Jun 23 01:42:38 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:42:45.992913 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756cff4-75b3-4f48-93d1-a4f863a6a536 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:45.993821 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756cff4-75b3-4f48-93d1-a4f863a6a536 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:45.993821 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756cff4-75b3-4f48-93d1-a4f863a6a536 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:46.000902 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1/2] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:42:45 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:42:46.006634 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2680fcd-d524-42bc-a8b6-8c029ed5acbd None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:46.007065 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2680fcd-d524-42bc-a8b6-8c029ed5acbd None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:46.007154 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2680fcd-d524-42bc-a8b6-8c029ed5acbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:46.076159 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: INFO dbcounter [None req-a2680fcd-d524-42bc-a8b6-8c029ed5acbd None None] Registered counter for database keystone Jun 23 01:42:46.100468 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG oslo_db.sqlalchemy.engines [None req-a2680fcd-d524-42bc-a8b6-8c029ed5acbd None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80460) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 23 01:42:46.111477 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writer thread running {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 23 01:42:46.298468 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG passlib.handlers.bcrypt [None req-a2680fcd-d524-42bc-a8b6-8c029ed5acbd None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80460) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 23 01:42:46.298894 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG passlib.handlers.bcrypt [None req-a2680fcd-d524-42bc-a8b6-8c029ed5acbd None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80460) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 23 01:42:46.354332 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-a2680fcd-d524-42bc-a8b6-8c029ed5acbd None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:42:46.355958 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-a2680fcd-d524-42bc-a8b6-8c029ed5acbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:42:46.483672 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 2/3] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:42:46 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 478 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 23 01:42:46.680465 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1756cff4-75b3-4f48-93d1-a4f863a6a536 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:42:46.757122 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: INFO dbcounter [None req-1756cff4-75b3-4f48-93d1-a4f863a6a536 None None] Registered counter for database keystone Jun 23 01:42:46.777960 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG oslo_db.sqlalchemy.engines [None req-1756cff4-75b3-4f48-93d1-a4f863a6a536 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80459) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 23 01:42:46.789075 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writer thread running {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 23 01:42:46.914580 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1756cff4-75b3-4f48-93d1-a4f863a6a536 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:42:46.920164 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326eb4f0-e1a5-4791-aaaa-9858943fb95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:42:46.920826 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326eb4f0-e1a5-4791-aaaa-9858943fb95c admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:46.921114 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326eb4f0-e1a5-4791-aaaa-9858943fb95c admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:46.921386 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326eb4f0-e1a5-4791-aaaa-9858943fb95c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:46.930422 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326eb4f0-e1a5-4791-aaaa-9858943fb95c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:42:46.981892 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: WARNING keystone.common.rbac_enforcer.enforcer [None req-326eb4f0-e1a5-4791-aaaa-9858943fb95c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 23 01:42:46.982160 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326eb4f0-e1a5-4791-aaaa-9858943fb95c admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:42:46.992885 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 2/4] 10.222.0.26 () {58 vars in 1324 bytes} [Thu Jun 23 01:42:46 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 01:42:47.001674 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2680fcd-d524-42bc-a8b6-8c029ed5acbd None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:42:47.017179 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2680fcd-d524-42bc-a8b6-8c029ed5acbd None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:42:47.023424 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf1e6b5-0043-46c1-abec-4a2e5a486106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:42:47.024560 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf1e6b5-0043-46c1-abec-4a2e5a486106 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:47.024848 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf1e6b5-0043-46c1-abec-4a2e5a486106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:47.025121 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf1e6b5-0043-46c1-abec-4a2e5a486106 admin admin] PATH_INFO: `/v3/projects/cc3d5c56f68947b3b757a0c2a92187ac` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:47.029441 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf1e6b5-0043-46c1-abec-4a2e5a486106 admin admin] RBAC: Authorizing `identity:get_project(project_id=cc3d5c56f68947b3b757a0c2a92187ac)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:42:47.079896 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: WARNING keystone.common.rbac_enforcer.enforcer [None req-dbf1e6b5-0043-46c1-abec-4a2e5a486106 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 23 01:42:47.080228 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf1e6b5-0043-46c1-abec-4a2e5a486106 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:42:47.084931 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 3/5] 10.222.0.26 () {56 vars in 1207 bytes} [Thu Jun 23 01:42:47 2022] GET /identity/v3/projects/cc3d5c56f68947b3b757a0c2a92187ac => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:42:49.064106 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1bbdc9-82ae-42ac-836b-969f9e774dbd None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:49.064675 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1bbdc9-82ae-42ac-836b-969f9e774dbd None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:49.064675 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1bbdc9-82ae-42ac-836b-969f9e774dbd None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:49.068879 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 3/6] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:42:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 01:42:49.074559 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf487b6-3ba2-431d-9759-e9f058123f70 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:49.074991 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf487b6-3ba2-431d-9759-e9f058123f70 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:49.075082 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf487b6-3ba2-431d-9759-e9f058123f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:49.095957 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-fbf487b6-3ba2-431d-9759-e9f058123f70 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:42:49.097455 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-fbf487b6-3ba2-431d-9759-e9f058123f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:42:49.106488 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 4/7] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:42:49 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 23 01:42:49.129078 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1bbdc9-82ae-42ac-836b-969f9e774dbd None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:42:49.150084 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1bbdc9-82ae-42ac-836b-969f9e774dbd None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:42:49.156059 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919267c0-7b63-4038-9a1b-a3dafd763189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:42:49.156683 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919267c0-7b63-4038-9a1b-a3dafd763189 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:49.156968 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919267c0-7b63-4038-9a1b-a3dafd763189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:49.157246 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919267c0-7b63-4038-9a1b-a3dafd763189 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:49.165991 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919267c0-7b63-4038-9a1b-a3dafd763189 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:42:49.167818 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919267c0-7b63-4038-9a1b-a3dafd763189 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:42:49.177289 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 4/8] 10.222.0.26 () {58 vars in 1324 bytes} [Thu Jun 23 01:42:49 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 01:42:49.183471 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf487b6-3ba2-431d-9759-e9f058123f70 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:42:49.192920 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf487b6-3ba2-431d-9759-e9f058123f70 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:42:49.198191 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44528827-8752-4005-9a2e-f5d20ea209b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:42:49.199055 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44528827-8752-4005-9a2e-f5d20ea209b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:49.199372 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44528827-8752-4005-9a2e-f5d20ea209b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:49.199644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44528827-8752-4005-9a2e-f5d20ea209b0 admin admin] PATH_INFO: `/v3/users/34e5aabcf8f5425f9c9f312f635ff77a` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:49.201128 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44528827-8752-4005-9a2e-f5d20ea209b0 admin admin] RBAC: Authorizing `identity:get_user(user_id=34e5aabcf8f5425f9c9f312f635ff77a)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:42:49.202979 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44528827-8752-4005-9a2e-f5d20ea209b0 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:42:49.204570 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 5/9] 10.222.0.26 () {56 vars in 1198 bytes} [Thu Jun 23 01:42:49 2022] GET /identity/v3/users/34e5aabcf8f5425f9c9f312f635ff77a => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:42:51.254050 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab3839-6b94-448a-9c09-db0b6275a7d0 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:51.254599 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab3839-6b94-448a-9c09-db0b6275a7d0 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:51.254599 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab3839-6b94-448a-9c09-db0b6275a7d0 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:51.256192 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 5/10] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:42:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:42:51.261549 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749265e-d93d-42cf-a3a7-8a9b68c14fb3 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:51.262023 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749265e-d93d-42cf-a3a7-8a9b68c14fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:51.262108 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749265e-d93d-42cf-a3a7-8a9b68c14fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:51.280185 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-d749265e-d93d-42cf-a3a7-8a9b68c14fb3 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:42:51.281472 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-d749265e-d93d-42cf-a3a7-8a9b68c14fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:42:51.295560 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b5251-7abd-4460-ab96-7b68b84c15af None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:51.295848 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b5251-7abd-4460-ab96-7b68b84c15af None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:51.296127 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b5251-7abd-4460-ab96-7b68b84c15af None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:51.297529 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 6/11] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:42:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 01:42:51.302560 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8059234-880a-4a3c-8f29-23c3b477b08f None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:51.302842 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8059234-880a-4a3c-8f29-23c3b477b08f None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:51.303152 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8059234-880a-4a3c-8f29-23c3b477b08f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:51.343367 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG passlib.handlers.bcrypt [None req-f8059234-880a-4a3c-8f29-23c3b477b08f None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80459) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 23 01:42:51.343799 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG passlib.handlers.bcrypt [None req-f8059234-880a-4a3c-8f29-23c3b477b08f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80459) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 23 01:42:51.368486 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 6/12] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:42:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:42:51.388758 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-f8059234-880a-4a3c-8f29-23c3b477b08f None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:42:51.390314 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-f8059234-880a-4a3c-8f29-23c3b477b08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:42:51.391523 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d749265e-d93d-42cf-a3a7-8a9b68c14fb3 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:42:51.416802 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:42:51.490627 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 7/13] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:42:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 01:42:51.500438 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d749265e-d93d-42cf-a3a7-8a9b68c14fb3 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:42:51.512496 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8059234-880a-4a3c-8f29-23c3b477b08f None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:42:51.534791 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b4c958-4bdd-42f0-a752-4eb4f9f94bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:42:51.535619 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b4c958-4bdd-42f0-a752-4eb4f9f94bc9 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:51.535902 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b4c958-4bdd-42f0-a752-4eb4f9f94bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:51.536176 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b4c958-4bdd-42f0-a752-4eb4f9f94bc9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:51.547841 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b4c958-4bdd-42f0-a752-4eb4f9f94bc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:42:51.559398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-c1b4c958-4bdd-42f0-a752-4eb4f9f94bc9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:42:51.559398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:42:51.559398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:42:51.559398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:42:51.559398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:42:51.559398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:42:51.559398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:42:51.559398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:42:51.559398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:42:51.559398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:42:51.559398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:42:51.559398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:42:51.559398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:42:51.559398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:42:51.559398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:42:51.559398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:42:51.560486 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:42:51.560486 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:42:51.560486 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:42:51.560486 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:42:51.560486 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:42:51.560486 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:42:51.560486 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:42:51.560486 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:42:51.560486 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:42:51.560486 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:42:51.560486 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:42:51.560486 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:42:51.560486 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:42:51.560486 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:42:51.560486 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:42:51.560486 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:42:51.561511 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:42:51.561511 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:42:51.561511 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:42:51.561511 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:42:51.561511 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:42:51.561511 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:42:51.561511 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:42:51.561511 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:42:51.561511 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:42:51.561511 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:42:51.561511 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:42:51.561511 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:42:51.561511 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 7/14] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jun 23 01:42:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:42:51.567953 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b4c958-4bdd-42f0-a752-4eb4f9f94bc9 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:42:51.618013 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8059234-880a-4a3c-8f29-23c3b477b08f None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:42:51.620437 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:42:51.648817 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:42:51.656073 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee782b6-9d32-42d3-ae8a-de384cf5ed5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:42:51.656986 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee782b6-9d32-42d3-ae8a-de384cf5ed5d None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:51.657276 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee782b6-9d32-42d3-ae8a-de384cf5ed5d None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:51.657556 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee782b6-9d32-42d3-ae8a-de384cf5ed5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:51.666996 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b4c958-4bdd-42f0-a752-4eb4f9f94bc9 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:42:51.700200 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0d28bd-3810-4ee5-bc7b-bac3cfd1fc65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:42:51.700842 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d28bd-3810-4ee5-bc7b-bac3cfd1fc65 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:51.701129 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d28bd-3810-4ee5-bc7b-bac3cfd1fc65 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:51.701400 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d28bd-3810-4ee5-bc7b-bac3cfd1fc65 None admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:51.702160 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0d28bd-3810-4ee5-bc7b-bac3cfd1fc65 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:42:51.702465 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0d28bd-3810-4ee5-bc7b-bac3cfd1fc65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:42:51.703936 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0d28bd-3810-4ee5-bc7b-bac3cfd1fc65 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:42:51.715726 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 8/15] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:42:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:42:51.721316 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0d28bd-3810-4ee5-bc7b-bac3cfd1fc65 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:42:51.725507 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee782b6-9d32-42d3-ae8a-de384cf5ed5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:42:51.727040 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee782b6-9d32-42d3-ae8a-de384cf5ed5d None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:42:51.822518 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0d28bd-3810-4ee5-bc7b-bac3cfd1fc65 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:42:51.826318 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 8/16] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:42:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 01:42:51.831930 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee782b6-9d32-42d3-ae8a-de384cf5ed5d None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:42:51.834819 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:42:51.856221 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07166418-91fa-4d30-a89b-a962b3bfe931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:42:51.857088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07166418-91fa-4d30-a89b-a962b3bfe931 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:51.857369 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07166418-91fa-4d30-a89b-a962b3bfe931 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:51.857642 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07166418-91fa-4d30-a89b-a962b3bfe931 None admin] PATH_INFO: `/v3/users/34e5aabcf8f5425f9c9f312f635ff77a` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:51.858854 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07166418-91fa-4d30-a89b-a962b3bfe931 None admin] RBAC: Authorizing `identity:get_user(user_id=34e5aabcf8f5425f9c9f312f635ff77a)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:42:51.860366 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07166418-91fa-4d30-a89b-a962b3bfe931 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:42:51.862017 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 9/17] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:42:51 2022] GET /identity/v3/users/34e5aabcf8f5425f9c9f312f635ff77a => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:42:51.867361 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07166418-91fa-4d30-a89b-a962b3bfe931 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:42:51.881723 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:42:51.929390 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee782b6-9d32-42d3-ae8a-de384cf5ed5d None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:42:51.968508 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679fd89c-0ab9-40f5-9c3e-52c6b607b91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:42:51.969399 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679fd89c-0ab9-40f5-9c3e-52c6b607b91f None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:51.969685 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679fd89c-0ab9-40f5-9c3e-52c6b607b91f None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:51.969955 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679fd89c-0ab9-40f5-9c3e-52c6b607b91f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:51.972300 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07166418-91fa-4d30-a89b-a962b3bfe931 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:42:51.989956 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679fd89c-0ab9-40f5-9c3e-52c6b607b91f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:42:51.991487 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679fd89c-0ab9-40f5-9c3e-52c6b607b91f None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:42:52.006724 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55df5583-6560-4dc9-b564-9c7c46dfaa2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:42:52.007541 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df5583-6560-4dc9-b564-9c7c46dfaa2e None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:52.007820 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df5583-6560-4dc9-b564-9c7c46dfaa2e None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:52.008093 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df5583-6560-4dc9-b564-9c7c46dfaa2e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:52.009283 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55df5583-6560-4dc9-b564-9c7c46dfaa2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:42:52.010721 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55df5583-6560-4dc9-b564-9c7c46dfaa2e None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:42:52.011883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-679fd89c-0ab9-40f5-9c3e-52c6b607b91f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:42:52.011883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:42:52.011883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:42:52.011883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:42:52.011883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:42:52.011883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:42:52.011883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:42:52.011883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:42:52.011883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:42:52.011883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:42:52.011883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:42:52.011883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:42:52.011883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:42:52.011883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:42:52.011883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:42:52.012844 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:42:52.012844 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:42:52.012844 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:42:52.012844 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:42:52.012844 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:42:52.012844 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:42:52.012844 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:42:52.012844 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:42:52.012844 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:42:52.012844 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:42:52.012844 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:42:52.012844 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:42:52.012844 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:42:52.012844 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:42:52.012844 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:42:52.012844 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:42:52.012844 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:42:52.014486 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:42:52.014486 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:42:52.014486 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:42:52.014486 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:42:52.014749 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 10/18] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:42:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:42:52.014825 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 9/19] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:42:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:42:52.018807 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55df5583-6560-4dc9-b564-9c7c46dfaa2e None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:42:52.019007 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679fd89c-0ab9-40f5-9c3e-52c6b607b91f None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:42:52.075802 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:42:52.097611 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:42:52.118119 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55df5583-6560-4dc9-b564-9c7c46dfaa2e None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:42:52.118530 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679fd89c-0ab9-40f5-9c3e-52c6b607b91f None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:42:52.150374 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7b5c51-806e-4485-9ec3-3c55bcf47a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:42:52.151026 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b5c51-806e-4485-9ec3-3c55bcf47a6a None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:52.151337 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b5c51-806e-4485-9ec3-3c55bcf47a6a None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:52.151612 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b5c51-806e-4485-9ec3-3c55bcf47a6a None admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:52.152087 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6520c758-8cf9-4c94-8c6b-49d12bb89c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:42:52.152335 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7b5c51-806e-4485-9ec3-3c55bcf47a6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:42:52.152639 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7b5c51-806e-4485-9ec3-3c55bcf47a6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:42:52.152754 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6520c758-8cf9-4c94-8c6b-49d12bb89c64 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:52.153041 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6520c758-8cf9-4c94-8c6b-49d12bb89c64 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:52.153325 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6520c758-8cf9-4c94-8c6b-49d12bb89c64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:52.154029 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7b5c51-806e-4485-9ec3-3c55bcf47a6a None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:42:52.154206 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6520c758-8cf9-4c94-8c6b-49d12bb89c64 None admin] RBAC: Adding query filter params (role.id=7b20d65f49ee4a48894b6c060bb5d580, scope.domain.id=default, user.id=34e5aabcf8f5425f9c9f312f635ff77a) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:42:52.154523 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6520c758-8cf9-4c94-8c6b-49d12bb89c64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:42:52.156018 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6520c758-8cf9-4c94-8c6b-49d12bb89c64 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:42:52.164586 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 10/20] 10.222.0.26 () {56 vars in 1322 bytes} [Thu Jun 23 01:42:52 2022] GET /identity/v3/role_assignments?user.id=34e5aabcf8f5425f9c9f312f635ff77a&scope.domain.id=default&role.id=7b20d65f49ee4a48894b6c060bb5d580 => generated 233 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:42:52.172267 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 11/21] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:42:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:42:54.531176 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff28758-060e-447e-8b1a-3f39c7fa7539 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:54.531727 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff28758-060e-447e-8b1a-3f39c7fa7539 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:54.531800 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff28758-060e-447e-8b1a-3f39c7fa7539 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:54.533150 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 11/22] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:42:54.551774 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36266a-5ef4-4e02-9ba8-f9de0bbc7ab6 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:54.552377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36266a-5ef4-4e02-9ba8-f9de0bbc7ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:54.552869 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36266a-5ef4-4e02-9ba8-f9de0bbc7ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:54.560456 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690c67f-979f-4314-9607-a0e71b6149d6 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:54.560749 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690c67f-979f-4314-9607-a0e71b6149d6 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:54.561028 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690c67f-979f-4314-9607-a0e71b6149d6 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:54.562589 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 12/23] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 01:42:54.572506 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f7cb17-c252-45f8-8a49-4d4bf8b7a085 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:54.572797 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f7cb17-c252-45f8-8a49-4d4bf8b7a085 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:54.573082 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f7cb17-c252-45f8-8a49-4d4bf8b7a085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:54.580204 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-ed36266a-5ef4-4e02-9ba8-f9de0bbc7ab6 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:42:54.581588 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-ed36266a-5ef4-4e02-9ba8-f9de0bbc7ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:42:54.608231 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-20f7cb17-c252-45f8-8a49-4d4bf8b7a085 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:42:54.609470 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-20f7cb17-c252-45f8-8a49-4d4bf8b7a085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:42:54.700561 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 12/24] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:42:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:42:54.749755 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 13/25] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:42:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 01:42:54.758501 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed36266a-5ef4-4e02-9ba8-f9de0bbc7ab6 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:42:54.814491 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f7cb17-c252-45f8-8a49-4d4bf8b7a085 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:42:54.861061 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:42:54.897498 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:42:54.899157 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed36266a-5ef4-4e02-9ba8-f9de0bbc7ab6 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:42:54.936764 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690780ed-0f04-4434-aa7e-7bdcec8eb82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:42:54.937558 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690780ed-0f04-4434-aa7e-7bdcec8eb82b None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:54.937840 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690780ed-0f04-4434-aa7e-7bdcec8eb82b None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:54.938112 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690780ed-0f04-4434-aa7e-7bdcec8eb82b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:54.949542 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690780ed-0f04-4434-aa7e-7bdcec8eb82b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:42:54.951153 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f7cb17-c252-45f8-8a49-4d4bf8b7a085 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:42:54.957506 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-690780ed-0f04-4434-aa7e-7bdcec8eb82b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:42:54.957506 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:42:54.957506 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:42:54.957506 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:42:54.957506 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:42:54.957506 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:42:54.957506 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:42:54.957506 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:42:54.957506 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:42:54.957506 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:42:54.957506 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:42:54.957506 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:42:54.957506 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:42:54.957506 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:42:54.957506 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:42:54.957506 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:42:54.958404 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:42:54.958404 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:42:54.958404 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:42:54.958404 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:42:54.958404 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:42:54.958404 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:42:54.958404 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:42:54.958404 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:42:54.958404 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:42:54.958404 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:42:54.958404 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:42:54.958404 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:42:54.958404 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:42:54.958404 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:42:54.958404 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:42:54.958404 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:42:54.959664 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:42:54.959664 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:42:54.959664 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:42:54.959664 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:42:54.959664 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:42:54.959664 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:42:54.959664 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:42:54.959664 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:42:54.959664 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:42:54.959664 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:42:54.959664 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:42:54.959664 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:42:54.959664 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 13/26] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jun 23 01:42:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:42:54.960514 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ed93d-ff1d-44d6-b410-260b4cce2619 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:54.960514 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ed93d-ff1d-44d6-b410-260b4cce2619 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:54.960776 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ed93d-ff1d-44d6-b410-260b4cce2619 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:54.962060 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 14/27] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:42:54.963104 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cbb062-7e27-4102-9914-86ab62dae934 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:54.963587 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cbb062-7e27-4102-9914-86ab62dae934 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:54.964284 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cbb062-7e27-4102-9914-86ab62dae934 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:54.965943 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 15/28] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:42:54.971110 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cbb062-7e27-4102-9914-86ab62dae934 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:42:54.997114 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a83a25-bfee-4c53-81fe-04ab24556499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:42:54.998071 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a83a25-bfee-4c53-81fe-04ab24556499 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:54.998451 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a83a25-bfee-4c53-81fe-04ab24556499 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:54.998823 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a83a25-bfee-4c53-81fe-04ab24556499 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:55.009601 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a83a25-bfee-4c53-81fe-04ab24556499 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:42:55.011399 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a83a25-bfee-4c53-81fe-04ab24556499 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:42:55.029400 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-89a83a25-bfee-4c53-81fe-04ab24556499 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:42:55.029400 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:42:55.029400 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:42:55.029400 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:42:55.029400 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:42:55.029400 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:42:55.029400 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:42:55.029400 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:42:55.029400 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:42:55.029400 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:42:55.029400 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:42:55.029400 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:42:55.029400 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:42:55.029400 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:42:55.029400 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:42:55.030204 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:42:55.030204 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:42:55.030204 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:42:55.030204 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:42:55.030204 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:42:55.030204 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:42:55.030204 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:42:55.030204 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:42:55.030204 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:42:55.030204 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:42:55.030204 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:42:55.030204 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:42:55.030204 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:42:55.030204 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:42:55.030204 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:42:55.030204 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:42:55.030204 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:42:55.030990 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:42:55.030990 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:42:55.030990 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:42:55.030990 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:42:55.033377 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 14/29] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:42:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:42:55.034994 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87673ea8-960d-419f-a25a-93c49b71be8a None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:55.035502 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87673ea8-960d-419f-a25a-93c49b71be8a None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:55.035964 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87673ea8-960d-419f-a25a-93c49b71be8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:55.053210 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-87673ea8-960d-419f-a25a-93c49b71be8a None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:42:55.054774 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-87673ea8-960d-419f-a25a-93c49b71be8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:42:55.083250 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cbb062-7e27-4102-9914-86ab62dae934 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:42:55.115389 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:42:55.122058 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53698577-1701-4fc2-95f3-d0c8a1ee7d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:42:55.123125 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53698577-1701-4fc2-95f3-d0c8a1ee7d08 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:55.123587 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53698577-1701-4fc2-95f3-d0c8a1ee7d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:55.124107 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53698577-1701-4fc2-95f3-d0c8a1ee7d08 None admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:55.125033 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53698577-1701-4fc2-95f3-d0c8a1ee7d08 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:42:55.125493 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53698577-1701-4fc2-95f3-d0c8a1ee7d08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:42:55.128038 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53698577-1701-4fc2-95f3-d0c8a1ee7d08 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:42:55.133337 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 15/30] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:42:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:42:55.134810 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eadc2fc-b763-4944-b21f-f88512d2d213 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:55.135192 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eadc2fc-b763-4944-b21f-f88512d2d213 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:55.135585 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eadc2fc-b763-4944-b21f-f88512d2d213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:55.138922 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 16/31] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:42:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:42:55.140006 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53698577-1701-4fc2-95f3-d0c8a1ee7d08 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:42:55.168259 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-3eadc2fc-b763-4944-b21f-f88512d2d213 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:42:55.169803 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-3eadc2fc-b763-4944-b21f-f88512d2d213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:42:55.200876 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:42:55.243952 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 16/32] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:42:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:42:55.245160 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eadc2fc-b763-4944-b21f-f88512d2d213 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:42:55.246827 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53698577-1701-4fc2-95f3-d0c8a1ee7d08 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:42:55.294171 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c6753-4b33-44d0-b452-4c46a8abc200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:42:55.295121 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c6753-4b33-44d0-b452-4c46a8abc200 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:55.295612 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c6753-4b33-44d0-b452-4c46a8abc200 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:55.296098 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c6753-4b33-44d0-b452-4c46a8abc200 None admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:55.297140 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1c6753-4b33-44d0-b452-4c46a8abc200 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:42:55.297613 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1c6753-4b33-44d0-b452-4c46a8abc200 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:42:55.299199 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1c6753-4b33-44d0-b452-4c46a8abc200 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:42:55.312449 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 17/33] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:42:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:42:55.313539 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c6753-4b33-44d0-b452-4c46a8abc200 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:42:55.347698 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eadc2fc-b763-4944-b21f-f88512d2d213 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:42:55.375572 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:42:55.383945 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4792c980-720f-4629-98c6-92d3fc6b4ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:42:55.385006 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792c980-720f-4629-98c6-92d3fc6b4ddd None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:55.385423 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792c980-720f-4629-98c6-92d3fc6b4ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:55.385918 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792c980-720f-4629-98c6-92d3fc6b4ddd None admin] PATH_INFO: `/v3/users/34e5aabcf8f5425f9c9f312f635ff77a` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:55.387759 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4792c980-720f-4629-98c6-92d3fc6b4ddd None admin] RBAC: Authorizing `identity:get_user(user_id=34e5aabcf8f5425f9c9f312f635ff77a)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:42:55.389353 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4792c980-720f-4629-98c6-92d3fc6b4ddd None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:42:55.391768 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 17/34] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:42:55 2022] GET /identity/v3/users/34e5aabcf8f5425f9c9f312f635ff77a => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:42:55.392728 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4792c980-720f-4629-98c6-92d3fc6b4ddd None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:42:55.424640 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:42:55.426881 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c6753-4b33-44d0-b452-4c46a8abc200 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:42:55.463728 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8bcf74-f06b-4d9b-8f84-30da7f2916fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:42:55.464715 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8bcf74-f06b-4d9b-8f84-30da7f2916fd None admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:55.465224 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8bcf74-f06b-4d9b-8f84-30da7f2916fd None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:55.465630 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8bcf74-f06b-4d9b-8f84-30da7f2916fd None admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:55.466542 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8bcf74-f06b-4d9b-8f84-30da7f2916fd None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:42:55.468177 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8bcf74-f06b-4d9b-8f84-30da7f2916fd None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:42:55.503695 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4792c980-720f-4629-98c6-92d3fc6b4ddd None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:42:55.538582 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a3381-a5f2-4c1c-b59d-ea3a73ee3ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:42:55.540025 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a3381-a5f2-4c1c-b59d-ea3a73ee3ffc None admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:55.540503 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a3381-a5f2-4c1c-b59d-ea3a73ee3ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:55.540912 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a3381-a5f2-4c1c-b59d-ea3a73ee3ffc None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:55.541803 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9a3381-a5f2-4c1c-b59d-ea3a73ee3ffc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:42:55.543327 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9a3381-a5f2-4c1c-b59d-ea3a73ee3ffc None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:42:55.601444 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 18/35] 10.222.0.26 () {60 vars in 1131 bytes} [Thu Jun 23 01:42:55 2022] POST /identity/v3/roles => generated 219 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:42:55.602968 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8bcf74-f06b-4d9b-8f84-30da7f2916fd None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:42:55.709989 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8bcf74-f06b-4d9b-8f84-30da7f2916fd None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:42:55.723640 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:42:55.731912 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 18/36] 10.222.0.26 () {60 vars in 1131 bytes} [Thu Jun 23 01:42:55 2022] POST /identity/v3/roles => generated 225 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:42:55.734481 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a3381-a5f2-4c1c-b59d-ea3a73ee3ffc None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:42:55.748911 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf0d58e-a2fb-4eda-b846-f3130cb3c565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:42:55.749822 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf0d58e-a2fb-4eda-b846-f3130cb3c565 None admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:55.750387 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf0d58e-a2fb-4eda-b846-f3130cb3c565 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:55.750820 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf0d58e-a2fb-4eda-b846-f3130cb3c565 None admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:55.751714 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf0d58e-a2fb-4eda-b846-f3130cb3c565 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:42:55.753536 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf0d58e-a2fb-4eda-b846-f3130cb3c565 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:42:55.802444 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 19/37] 10.222.0.26 () {60 vars in 1141 bytes} [Thu Jun 23 01:42:55 2022] POST /identity/v3/projects => generated 301 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:42:55.910213 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:42:55.921564 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a3381-a5f2-4c1c-b59d-ea3a73ee3ffc None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:42:55.956772 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee53a077-f0ac-442d-96a6-038001bf82b7 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:55.957431 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee53a077-f0ac-442d-96a6-038001bf82b7 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:55.957899 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee53a077-f0ac-442d-96a6-038001bf82b7 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:55.959677 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 20/38] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:42:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 01:42:55.969070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75b745-3d79-4351-a10b-a78aff90619f None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:55.969698 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75b745-3d79-4351-a10b-a78aff90619f None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:55.970175 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75b745-3d79-4351-a10b-a78aff90619f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:55.971282 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7facfb61-931f-4554-b8ac-d732d899ecac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:42:55.974864 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7facfb61-931f-4554-b8ac-d732d899ecac None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:55.974864 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7facfb61-931f-4554-b8ac-d732d899ecac None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:55.974864 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7facfb61-931f-4554-b8ac-d732d899ecac None admin] PATH_INFO: `/v3/domains/default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:55.974864 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7facfb61-931f-4554-b8ac-d732d899ecac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:42:55.975819 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7facfb61-931f-4554-b8ac-d732d899ecac None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:42:55.977722 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 19/39] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:42:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:42:55.983628 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7facfb61-931f-4554-b8ac-d732d899ecac None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:42:56.012446 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-ba75b745-3d79-4351-a10b-a78aff90619f None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:42:56.014455 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-ba75b745-3d79-4351-a10b-a78aff90619f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:42:56.097972 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7facfb61-931f-4554-b8ac-d732d899ecac None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:42:56.138505 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c4832-b24f-4ec0-ab2e-8d303cf12fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:42:56.139425 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c4832-b24f-4ec0-ab2e-8d303cf12fce None admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:56.139708 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c4832-b24f-4ec0-ab2e-8d303cf12fce None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:56.139991 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c4832-b24f-4ec0-ab2e-8d303cf12fce None admin] PATH_INFO: `/v3/domains/default/users/34e5aabcf8f5425f9c9f312f635ff77a/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:56.142215 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770c4832-b24f-4ec0-ab2e-8d303cf12fce None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=34e5aabcf8f5425f9c9f312f635ff77a, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:42:56.143738 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770c4832-b24f-4ec0-ab2e-8d303cf12fce None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:42:56.149252 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 21/40] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:42:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:42:56.150518 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409644b-f0f3-4a23-8e19-6089e133ec15 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:56.150811 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409644b-f0f3-4a23-8e19-6089e133ec15 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:56.151089 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409644b-f0f3-4a23-8e19-6089e133ec15 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:56.153861 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 22/41] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:42:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:42:56.158649 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb8bad-d963-4a6b-b419-c3938734c2c2 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:56.158931 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb8bad-d963-4a6b-b419-c3938734c2c2 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:56.159239 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb8bad-d963-4a6b-b419-c3938734c2c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:56.166039 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 20/42] 10.222.0.26 () {58 vars in 1361 bytes} [Thu Jun 23 01:42:55 2022] PUT /identity/v3/domains/default/users/34e5aabcf8f5425f9c9f312f635ff77a/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:42:56.183902 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c4832-b24f-4ec0-ab2e-8d303cf12fce None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:42:56.185167 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-95fb8bad-d963-4a6b-b419-c3938734c2c2 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:42:56.186582 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-95fb8bad-d963-4a6b-b419-c3938734c2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:42:56.210932 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:42:56.315081 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c4832-b24f-4ec0-ab2e-8d303cf12fce None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:42:56.361374 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a74e0f8-3bb0-45e9-87e8-eb2f2efeef50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:42:56.362059 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74e0f8-3bb0-45e9-87e8-eb2f2efeef50 None admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:56.362340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74e0f8-3bb0-45e9-87e8-eb2f2efeef50 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:56.362617 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74e0f8-3bb0-45e9-87e8-eb2f2efeef50 None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:56.363461 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a74e0f8-3bb0-45e9-87e8-eb2f2efeef50 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:42:56.364845 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a74e0f8-3bb0-45e9-87e8-eb2f2efeef50 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:42:56.375990 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 23/43] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:42:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 01:42:56.384779 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 21/44] 10.222.0.26 () {60 vars in 1131 bytes} [Thu Jun 23 01:42:56 2022] POST /identity/v3/roles => generated 223 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 01:42:56.399556 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb8bad-d963-4a6b-b419-c3938734c2c2 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:42:56.428173 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:42:56.516521 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb8bad-d963-4a6b-b419-c3938734c2c2 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:42:56.554331 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8380be-f94c-4465-b5e6-07ff37d5b1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:42:56.555150 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8380be-f94c-4465-b5e6-07ff37d5b1de None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:56.555541 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8380be-f94c-4465-b5e6-07ff37d5b1de None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:56.556117 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8380be-f94c-4465-b5e6-07ff37d5b1de None admin] PATH_INFO: `/v3/domains/default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:56.557378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8380be-f94c-4465-b5e6-07ff37d5b1de None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:42:56.558853 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8380be-f94c-4465-b5e6-07ff37d5b1de None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:42:56.560539 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 24/45] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:42:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:42:56.566133 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a74e0f8-3bb0-45e9-87e8-eb2f2efeef50 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:42:56.648605 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:42:56.671503 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a74e0f8-3bb0-45e9-87e8-eb2f2efeef50 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:42:56.708625 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5310986-e722-4fe0-b9d9-a12b1a9ecb74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:42:56.709344 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5310986-e722-4fe0-b9d9-a12b1a9ecb74 None admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:56.709634 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5310986-e722-4fe0-b9d9-a12b1a9ecb74 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:56.709893 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5310986-e722-4fe0-b9d9-a12b1a9ecb74 None admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:56.710629 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5310986-e722-4fe0-b9d9-a12b1a9ecb74 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:42:56.712119 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5310986-e722-4fe0-b9d9-a12b1a9ecb74 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:42:56.740946 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 22/46] 10.222.0.26 () {60 vars in 1141 bytes} [Thu Jun 23 01:42:56 2022] POST /identity/v3/projects => generated 312 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:42:58.186919 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710befa8-69c3-47f3-bd18-f53595cdc40f None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:58.187487 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710befa8-69c3-47f3-bd18-f53595cdc40f None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:58.187487 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710befa8-69c3-47f3-bd18-f53595cdc40f None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:58.188873 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 25/47] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:42:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:42:58.194205 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53c47b-99ad-405a-ae17-4fcbaa91fc1e None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:58.194640 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53c47b-99ad-405a-ae17-4fcbaa91fc1e None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:58.194722 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53c47b-99ad-405a-ae17-4fcbaa91fc1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:58.211802 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-6e53c47b-99ad-405a-ae17-4fcbaa91fc1e None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:42:58.213318 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-6e53c47b-99ad-405a-ae17-4fcbaa91fc1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:42:58.279581 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 23/48] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:42:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:42:58.302393 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710befa8-69c3-47f3-bd18-f53595cdc40f None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:42:58.396892 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:42:58.408303 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710befa8-69c3-47f3-bd18-f53595cdc40f None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:42:58.444895 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e8bc40-b893-4da0-8625-462e5462f426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:42:58.445615 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e8bc40-b893-4da0-8625-462e5462f426 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:58.445902 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e8bc40-b893-4da0-8625-462e5462f426 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:58.446181 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e8bc40-b893-4da0-8625-462e5462f426 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:58.457628 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e8bc40-b893-4da0-8625-462e5462f426 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:42:58.465525 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-01e8bc40-b893-4da0-8625-462e5462f426 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:42:58.465525 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:42:58.465525 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:42:58.465525 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:42:58.465525 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:42:58.465525 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:42:58.465525 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:42:58.465525 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:42:58.465525 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:42:58.465525 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:42:58.465525 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:42:58.465525 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:42:58.465525 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:42:58.465525 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:42:58.465525 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:42:58.465525 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:42:58.466504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:42:58.466504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:42:58.466504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:42:58.466504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:42:58.466504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:42:58.466504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:42:58.466504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:42:58.466504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:42:58.466504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:42:58.466504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:42:58.466504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:42:58.466504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:42:58.466504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:42:58.466504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:42:58.466504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:42:58.466504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:42:58.472071 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:42:58.472071 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:42:58.472071 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:42:58.472071 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:42:58.472071 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:42:58.472071 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:42:58.472071 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:42:58.472071 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:42:58.472071 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:42:58.472071 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:42:58.472071 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:42:58.472071 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:42:58.472071 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 26/49] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jun 23 01:42:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:42:58.472698 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e53c47b-99ad-405a-ae17-4fcbaa91fc1e None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:42:58.534977 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:42:58.578151 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e53c47b-99ad-405a-ae17-4fcbaa91fc1e None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:42:58.613758 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9237efcc-d0fe-4284-9d6a-4e6ad8ce80d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:42:58.614422 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9237efcc-d0fe-4284-9d6a-4e6ad8ce80d5 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:58.614702 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9237efcc-d0fe-4284-9d6a-4e6ad8ce80d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:58.614975 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9237efcc-d0fe-4284-9d6a-4e6ad8ce80d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:58.615799 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9237efcc-d0fe-4284-9d6a-4e6ad8ce80d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:42:58.616107 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9237efcc-d0fe-4284-9d6a-4e6ad8ce80d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:42:58.617510 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9237efcc-d0fe-4284-9d6a-4e6ad8ce80d5 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:42:58.629446 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 24/50] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:42:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:42:58.635174 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e8bc40-b893-4da0-8625-462e5462f426 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:42:58.733983 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c8b0e-9b99-407f-bf38-bbf15a1ef5ee None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:58.734164 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c8b0e-9b99-407f-bf38-bbf15a1ef5ee None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:58.734443 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c8b0e-9b99-407f-bf38-bbf15a1ef5ee None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:58.735857 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 25/51] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:42:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 01:42:58.736252 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e8bc40-b893-4da0-8625-462e5462f426 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:42:58.741308 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00718a6b-f2bc-4b17-b41d-093877ed1647 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:58.741600 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00718a6b-f2bc-4b17-b41d-093877ed1647 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:58.741900 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00718a6b-f2bc-4b17-b41d-093877ed1647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:58.758898 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-00718a6b-f2bc-4b17-b41d-093877ed1647 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:42:58.760185 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-00718a6b-f2bc-4b17-b41d-093877ed1647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:42:58.772811 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc520036-4f6f-4625-8a7a-93854484be13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:42:58.773690 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc520036-4f6f-4625-8a7a-93854484be13 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:58.773977 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc520036-4f6f-4625-8a7a-93854484be13 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:58.774252 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc520036-4f6f-4625-8a7a-93854484be13 None admin] PATH_INFO: `/v3/users/34e5aabcf8f5425f9c9f312f635ff77a` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:58.775508 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc520036-4f6f-4625-8a7a-93854484be13 None admin] RBAC: Authorizing `identity:get_user(user_id=34e5aabcf8f5425f9c9f312f635ff77a)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:42:58.776986 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc520036-4f6f-4625-8a7a-93854484be13 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:42:58.778644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 27/52] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:42:58 2022] GET /identity/v3/users/34e5aabcf8f5425f9c9f312f635ff77a => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:42:58.784175 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc520036-4f6f-4625-8a7a-93854484be13 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:42:58.808288 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:42:58.828336 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 26/53] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:42:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 01:42:58.851647 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00718a6b-f2bc-4b17-b41d-093877ed1647 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:42:58.884897 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc520036-4f6f-4625-8a7a-93854484be13 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:42:58.902741 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:42:58.917261 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5222be55-b85c-4c3b-8ce7-14a9ee23f988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:42:58.918055 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222be55-b85c-4c3b-8ce7-14a9ee23f988 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:58.918341 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222be55-b85c-4c3b-8ce7-14a9ee23f988 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:58.918621 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222be55-b85c-4c3b-8ce7-14a9ee23f988 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:58.919847 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5222be55-b85c-4c3b-8ce7-14a9ee23f988 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:42:58.921329 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5222be55-b85c-4c3b-8ce7-14a9ee23f988 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:42:58.922960 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 28/54] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:42:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:42:58.928608 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5222be55-b85c-4c3b-8ce7-14a9ee23f988 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:42:58.950172 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00718a6b-f2bc-4b17-b41d-093877ed1647 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:42:59.022860 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:42:59.037082 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5222be55-b85c-4c3b-8ce7-14a9ee23f988 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:42:59.073299 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690882ce-5b37-4e74-9b64-c2c48cc13a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:42:59.073956 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690882ce-5b37-4e74-9b64-c2c48cc13a5f None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:59.074238 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690882ce-5b37-4e74-9b64-c2c48cc13a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:59.074496 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690882ce-5b37-4e74-9b64-c2c48cc13a5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:59.075370 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690882ce-5b37-4e74-9b64-c2c48cc13a5f None admin] RBAC: Adding query filter params (role.id=7b20d65f49ee4a48894b6c060bb5d580, scope.domain.id=default, user.id=34e5aabcf8f5425f9c9f312f635ff77a) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:42:59.075686 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690882ce-5b37-4e74-9b64-c2c48cc13a5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:42:59.077164 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690882ce-5b37-4e74-9b64-c2c48cc13a5f None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:42:59.084482 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 29/55] 10.222.0.26 () {56 vars in 1322 bytes} [Thu Jun 23 01:42:58 2022] GET /identity/v3/role_assignments?user.id=34e5aabcf8f5425f9c9f312f635ff77a&scope.domain.id=default&role.id=7b20d65f49ee4a48894b6c060bb5d580 => generated 530 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:42:59.093407 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ad7db-c93e-4264-8a31-233ee963e639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:42:59.094235 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ad7db-c93e-4264-8a31-233ee963e639 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:59.094523 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ad7db-c93e-4264-8a31-233ee963e639 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:59.094802 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ad7db-c93e-4264-8a31-233ee963e639 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:59.096329 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ad7db-c93e-4264-8a31-233ee963e639 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:42:59.097883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ad7db-c93e-4264-8a31-233ee963e639 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:42:59.099760 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 27/56] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:42:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:42:59.105799 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690882ce-5b37-4e74-9b64-c2c48cc13a5f None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:42:59.217729 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690882ce-5b37-4e74-9b64-c2c48cc13a5f None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:42:59.254518 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f25f6-57ed-4231-bd98-f714477af0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:42:59.255182 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f25f6-57ed-4231-bd98-f714477af0a3 None admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:42:59.255493 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f25f6-57ed-4231-bd98-f714477af0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:42:59.255771 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f25f6-57ed-4231-bd98-f714477af0a3 None admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:42:59.256538 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2f25f6-57ed-4231-bd98-f714477af0a3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:42:59.258012 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2f25f6-57ed-4231-bd98-f714477af0a3 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:42:59.284760 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 30/57] 10.222.0.26 () {60 vars in 1140 bytes} [Thu Jun 23 01:42:59 2022] POST /identity/v3/projects => generated 298 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:43:01.283197 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b776486-3da6-41a6-8d8e-bdf57b7866bf None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:01.283753 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b776486-3da6-41a6-8d8e-bdf57b7866bf None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:01.283753 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b776486-3da6-41a6-8d8e-bdf57b7866bf None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:01.285180 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 28/58] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:01.291430 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b53834-4133-428f-aebc-111c8ded6afe None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:01.292111 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b53834-4133-428f-aebc-111c8ded6afe None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:01.292207 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b53834-4133-428f-aebc-111c8ded6afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:01.302170 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:01.314629 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-05b53834-4133-428f-aebc-111c8ded6afe None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:01.315872 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-05b53834-4133-428f-aebc-111c8ded6afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:01.391768 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 31/59] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:01.416367 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b776486-3da6-41a6-8d8e-bdf57b7866bf None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:01.520403 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b776486-3da6-41a6-8d8e-bdf57b7866bf None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:01.542296 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:01.557782 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40788041-a077-457c-927d-3bd4fb2972a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:01.558592 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40788041-a077-457c-927d-3bd4fb2972a7 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:01.558874 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40788041-a077-457c-927d-3bd4fb2972a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:01.559146 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40788041-a077-457c-927d-3bd4fb2972a7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:01.560437 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40788041-a077-457c-927d-3bd4fb2972a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:01.561878 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40788041-a077-457c-927d-3bd4fb2972a7 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:01.563526 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 29/60] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:43:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:01.571035 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b53834-4133-428f-aebc-111c8ded6afe None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:01.688023 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b53834-4133-428f-aebc-111c8ded6afe None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:01.722821 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f19fa-551f-4632-a376-87097fca3a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:01.723500 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f19fa-551f-4632-a376-87097fca3a3f None admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:01.723783 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f19fa-551f-4632-a376-87097fca3a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:01.724063 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f19fa-551f-4632-a376-87097fca3a3f None admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:01.724846 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428f19fa-551f-4632-a376-87097fca3a3f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:01.726299 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428f19fa-551f-4632-a376-87097fca3a3f None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:01.735017 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:01.757871 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 32/61] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Jun 23 01:43:01 2022] POST /identity/v3/users => generated 275 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:43:04.675115 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b055f8d-08fc-442c-8827-e537b61fd672 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:04.676154 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b055f8d-08fc-442c-8827-e537b61fd672 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:04.676633 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b055f8d-08fc-442c-8827-e537b61fd672 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:04.678546 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 30/62] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:04.683994 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7687c2-d3ef-47d8-a089-afbc2e4495af None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:04.684389 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7687c2-d3ef-47d8-a089-afbc2e4495af None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:04.684567 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7687c2-d3ef-47d8-a089-afbc2e4495af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:04.707302 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-0d7687c2-d3ef-47d8-a089-afbc2e4495af None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:04.708501 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-0d7687c2-d3ef-47d8-a089-afbc2e4495af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:04.804038 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 33/63] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:04.846779 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b055f8d-08fc-442c-8827-e537b61fd672 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:04.989938 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b055f8d-08fc-442c-8827-e537b61fd672 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:04.991392 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba4b149-44a2-48ca-9b0b-7f786a7a2adb None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:04.991979 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba4b149-44a2-48ca-9b0b-7f786a7a2adb None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:04.992576 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba4b149-44a2-48ca-9b0b-7f786a7a2adb None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:04.994025 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 34/64] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 01:43:05.001946 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c23ac5-157e-4b36-8119-dda6e10daed4 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:05.002524 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c23ac5-157e-4b36-8119-dda6e10daed4 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:05.002991 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c23ac5-157e-4b36-8119-dda6e10daed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:05.036395 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-e9c23ac5-157e-4b36-8119-dda6e10daed4 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:05.037651 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-e9c23ac5-157e-4b36-8119-dda6e10daed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:05.042148 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca13dc-f91a-44f1-b8f9-ac392041e219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:05.042972 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca13dc-f91a-44f1-b8f9-ac392041e219 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:05.043284 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca13dc-f91a-44f1-b8f9-ac392041e219 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:05.043563 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca13dc-f91a-44f1-b8f9-ac392041e219 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:05.071755 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca13dc-f91a-44f1-b8f9-ac392041e219 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:05.082098 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-e6ca13dc-f91a-44f1-b8f9-ac392041e219 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 01:43:05.082098 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:05.082098 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:05.082098 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:05.082098 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:05.082098 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:05.082098 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:05.082098 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:05.082098 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:05.082098 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:05.082098 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:05.082098 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:05.082098 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:05.082098 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:05.082098 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:05.082098 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:05.082953 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:05.082953 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:05.082953 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:05.082953 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:05.082953 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:05.082953 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:05.082953 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:05.082953 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:05.082953 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:05.082953 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:05.082953 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:05.082953 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:05.082953 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:05.082953 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:05.082953 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:05.082953 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:05.083748 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:05.083748 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:05.083748 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:05.083748 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:05.083748 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:05.083748 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:05.083748 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:05.083748 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:05.083748 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:05.083748 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:05.083748 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 01:43:05.083748 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:43:05.084473 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 31/65] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:43:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:43:05.090589 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca13dc-f91a-44f1-b8f9-ac392041e219 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:05.169203 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:05.173153 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 35/66] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 01:43:05.212389 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c23ac5-157e-4b36-8119-dda6e10daed4 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:05.224986 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca13dc-f91a-44f1-b8f9-ac392041e219 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:05.266752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9f15ac-f4fb-473f-a0bf-7e09995c88a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:05.270423 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f15ac-f4fb-473f-a0bf-7e09995c88a7 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:05.270848 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f15ac-f4fb-473f-a0bf-7e09995c88a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:05.271327 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f15ac-f4fb-473f-a0bf-7e09995c88a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:05.272225 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9f15ac-f4fb-473f-a0bf-7e09995c88a7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:05.272691 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9f15ac-f4fb-473f-a0bf-7e09995c88a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:05.274414 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9f15ac-f4fb-473f-a0bf-7e09995c88a7 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:05.289468 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 32/67] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jun 23 01:43:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:05.295241 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32855d43-8e91-46ea-8a00-48931422b251 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:05.295241 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32855d43-8e91-46ea-8a00-48931422b251 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:05.295241 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32855d43-8e91-46ea-8a00-48931422b251 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:05.295241 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 33/68] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:05.301554 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b3130b-2970-4cab-8ab0-6b50535ee541 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:05.301851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b3130b-2970-4cab-8ab0-6b50535ee541 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:05.302138 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b3130b-2970-4cab-8ab0-6b50535ee541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:05.318837 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:05.325546 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-24b3130b-2970-4cab-8ab0-6b50535ee541 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:05.326838 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-24b3130b-2970-4cab-8ab0-6b50535ee541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:05.371752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c23ac5-157e-4b36-8119-dda6e10daed4 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:05.399642 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 34/69] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:05.401352 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b3130b-2970-4cab-8ab0-6b50535ee541 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:05.411569 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b94b3be-fc3b-4b84-8f6c-efd640c094b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:05.412628 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94b3be-fc3b-4b84-8f6c-efd640c094b5 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:05.413054 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94b3be-fc3b-4b84-8f6c-efd640c094b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:05.413466 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94b3be-fc3b-4b84-8f6c-efd640c094b5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:05.424668 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b94b3be-fc3b-4b84-8f6c-efd640c094b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:05.432796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-3b94b3be-fc3b-4b84-8f6c-efd640c094b5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:43:05.432796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:05.432796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:05.432796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:05.432796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:05.432796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:05.432796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:05.432796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:05.432796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:05.432796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:05.432796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:05.432796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:05.432796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:05.432796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:05.432796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:05.432796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:05.433614 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:05.433614 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:05.433614 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:05.433614 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:05.433614 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:05.433614 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:05.433614 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:05.433614 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:05.433614 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:05.433614 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:05.433614 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:05.433614 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:05.433614 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:05.433614 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:05.433614 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:05.433614 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:05.434367 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:05.434367 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:05.434367 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:05.434367 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:05.434367 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:05.434367 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:05.434367 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:05.434367 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:05.434367 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:05.434367 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:05.434367 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:43:05.434367 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:43:05.434367 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 36/70] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jun 23 01:43:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:43:05.435503 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f459b-fcdf-499a-990d-c877a6c1e62f None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:05.435606 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f459b-fcdf-499a-990d-c877a6c1e62f None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:05.436044 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f459b-fcdf-499a-990d-c877a6c1e62f None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:05.437247 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 37/71] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:05.440308 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f459b-fcdf-499a-990d-c877a6c1e62f None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:05.456853 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:05.522897 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b3130b-2970-4cab-8ab0-6b50535ee541 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:05.547180 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f459b-fcdf-499a-990d-c877a6c1e62f None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:05.556808 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193c162-14bc-48ff-8341-9ebd4004acad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:05.557599 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:05.557705 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193c162-14bc-48ff-8341-9ebd4004acad None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:05.557965 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193c162-14bc-48ff-8341-9ebd4004acad None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:05.558251 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193c162-14bc-48ff-8341-9ebd4004acad None admin] PATH_INFO: `/v3/users/dce468546a1841c0bc2f6bda45393ccd` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:05.573733 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193c162-14bc-48ff-8341-9ebd4004acad None admin] RBAC: Authorizing `identity:get_user(user_id=dce468546a1841c0bc2f6bda45393ccd)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:05.575233 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193c162-14bc-48ff-8341-9ebd4004acad None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:05.576885 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 35/72] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:43:05 2022] GET /identity/v3/users/dce468546a1841c0bc2f6bda45393ccd => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:05.577672 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193c162-14bc-48ff-8341-9ebd4004acad None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:05.580344 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ca2d30-3b6b-4656-946a-805ab0b263f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:05.581599 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca2d30-3b6b-4656-946a-805ab0b263f5 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:05.581599 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca2d30-3b6b-4656-946a-805ab0b263f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:05.581765 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca2d30-3b6b-4656-946a-805ab0b263f5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:05.592228 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ca2d30-3b6b-4656-946a-805ab0b263f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:05.599953 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-00ca2d30-3b6b-4656-946a-805ab0b263f5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 01:43:05.599953 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:05.599953 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:05.599953 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:05.599953 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:05.599953 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:05.599953 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:05.599953 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:05.599953 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:05.599953 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:05.599953 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:05.599953 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:05.599953 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:05.599953 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:05.599953 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:05.599953 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:05.600770 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:05.600770 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:05.600770 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:05.600770 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:05.600770 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:05.600770 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:05.600770 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:05.600770 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:05.600770 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:05.600770 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:05.600770 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:05.600770 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:05.600770 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:05.600770 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:05.600770 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:05.600770 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:05.601529 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:05.601529 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:05.601529 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:05.601529 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:05.601529 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:05.601529 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:05.601529 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:05.601529 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:05.601529 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:05.601529 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:05.601529 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 01:43:05.601529 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:43:05.601529 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 38/73] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:43:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:43:05.602690 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b442793a-4ee9-4b3c-8798-ec25088e3c39 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:05.603196 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b442793a-4ee9-4b3c-8798-ec25088e3c39 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:05.603708 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b442793a-4ee9-4b3c-8798-ec25088e3c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:05.622386 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-b442793a-4ee9-4b3c-8798-ec25088e3c39 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:05.623647 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-b442793a-4ee9-4b3c-8798-ec25088e3c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:05.677334 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:05.681523 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193c162-14bc-48ff-8341-9ebd4004acad None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:05.691529 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 39/74] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:05.693314 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9e091-74f6-45bf-a6e8-7712a153e47a None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:05.693605 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9e091-74f6-45bf-a6e8-7712a153e47a None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:05.693885 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9e091-74f6-45bf-a6e8-7712a153e47a None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:05.695152 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 40/75] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:05.695873 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e9e091-74f6-45bf-a6e8-7712a153e47a None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:05.728223 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8f51a1-a5c8-4072-89f5-3baff7cfb133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:05.728898 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8f51a1-a5c8-4072-89f5-3baff7cfb133 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:05.729181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8f51a1-a5c8-4072-89f5-3baff7cfb133 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:05.729459 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8f51a1-a5c8-4072-89f5-3baff7cfb133 None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:05.730211 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8f51a1-a5c8-4072-89f5-3baff7cfb133 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:05.730522 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8f51a1-a5c8-4072-89f5-3baff7cfb133 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:05.731943 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8f51a1-a5c8-4072-89f5-3baff7cfb133 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:05.740203 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 36/76] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:43:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:05.740203 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8f51a1-a5c8-4072-89f5-3baff7cfb133 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:05.791723 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:05.795855 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e9e091-74f6-45bf-a6e8-7712a153e47a None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:05.830453 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa3ef06-4062-4772-a016-c1af3ba8b3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:05.831089 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa3ef06-4062-4772-a016-c1af3ba8b3f8 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:05.831404 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa3ef06-4062-4772-a016-c1af3ba8b3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:05.831681 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa3ef06-4062-4772-a016-c1af3ba8b3f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:05.844748 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8f51a1-a5c8-4072-89f5-3baff7cfb133 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:05.877536 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b15ae1-e96a-4685-b369-73d92904206a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:05.878202 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b15ae1-e96a-4685-b369-73d92904206a None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:05.878485 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b15ae1-e96a-4685-b369-73d92904206a None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:05.878760 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b15ae1-e96a-4685-b369-73d92904206a None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:05.879554 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b15ae1-e96a-4685-b369-73d92904206a None admin] RBAC: Adding query filter params (name=member) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:05.879864 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b15ae1-e96a-4685-b369-73d92904206a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:05.881302 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b15ae1-e96a-4685-b369-73d92904206a None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:05.889244 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 37/77] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jun 23 01:43:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:05.892586 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c63731-e92c-4bc1-b97c-b400c7f10c4d None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:05.892883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c63731-e92c-4bc1-b97c-b400c7f10c4d None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:05.893167 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c63731-e92c-4bc1-b97c-b400c7f10c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:05.900549 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa3ef06-4062-4772-a016-c1af3ba8b3f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:05.902049 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa3ef06-4062-4772-a016-c1af3ba8b3f8 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:05.909560 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-67c63731-e92c-4bc1-b97c-b400c7f10c4d None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:05.910762 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-67c63731-e92c-4bc1-b97c-b400c7f10c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:05.927345 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:05.978346 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 38/78] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:05.979163 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c63731-e92c-4bc1-b97c-b400c7f10c4d None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:05.999872 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:06.004539 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 41/79] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:43:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 01:43:06.005448 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa3ef06-4062-4772-a016-c1af3ba8b3f8 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:06.085771 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c63731-e92c-4bc1-b97c-b400c7f10c4d None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:06.104544 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa3ef06-4062-4772-a016-c1af3ba8b3f8 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:06.118488 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da20afc0-8c38-4f3f-b31f-796517d3fafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:06.119345 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20afc0-8c38-4f3f-b31f-796517d3fafd None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:06.119637 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20afc0-8c38-4f3f-b31f-796517d3fafd None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:06.119940 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20afc0-8c38-4f3f-b31f-796517d3fafd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:06.129908 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da20afc0-8c38-4f3f-b31f-796517d3fafd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:06.138010 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-da20afc0-8c38-4f3f-b31f-796517d3fafd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 01:43:06.138010 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:06.138010 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:06.138010 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:06.138010 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:06.138010 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:06.138010 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:06.138010 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:06.138010 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:06.138010 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:06.138010 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:06.138010 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:06.138010 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:06.138010 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:06.138010 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:06.138010 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:06.139018 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:06.139018 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:06.139018 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:06.139018 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:06.139018 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:06.139018 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:06.139018 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:06.139018 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:06.139018 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:06.139018 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:06.139018 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:06.139018 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:06.139018 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:06.139018 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:06.139018 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:06.139018 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:06.139828 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:06.139828 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:06.139828 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:06.139828 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:06.139828 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:06.139828 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:06.139828 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:06.139828 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:06.139828 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:06.139828 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:06.139828 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 01:43:06.139828 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:43:06.140405 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330cdb85-c90d-402e-9871-5f43164f15f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:06.140485 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 39/80] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Jun 23 01:43:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:43:06.140548 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330cdb85-c90d-402e-9871-5f43164f15f0 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:06.140548 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330cdb85-c90d-402e-9871-5f43164f15f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:06.140669 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da20afc0-8c38-4f3f-b31f-796517d3fafd None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:06.140732 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330cdb85-c90d-402e-9871-5f43164f15f0 None admin] PATH_INFO: `/v3/users/34e5aabcf8f5425f9c9f312f635ff77a` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:06.141592 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330cdb85-c90d-402e-9871-5f43164f15f0 None admin] RBAC: Authorizing `identity:get_user(user_id=34e5aabcf8f5425f9c9f312f635ff77a)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:06.143069 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330cdb85-c90d-402e-9871-5f43164f15f0 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:06.149003 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 42/81] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:43:06 2022] GET /identity/v3/users/34e5aabcf8f5425f9c9f312f635ff77a => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:06.149003 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330cdb85-c90d-402e-9871-5f43164f15f0 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:06.156698 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:06.210578 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:06.243920 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330cdb85-c90d-402e-9871-5f43164f15f0 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:06.244099 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da20afc0-8c38-4f3f-b31f-796517d3fafd None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:06.279405 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c691b79a-b3aa-4c93-a841-65da39c2d833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:06.280195 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691b79a-b3aa-4c93-a841-65da39c2d833 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:06.280473 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691b79a-b3aa-4c93-a841-65da39c2d833 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:06.280813 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691b79a-b3aa-4c93-a841-65da39c2d833 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:06.280951 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8183d-ed6e-4f60-a87a-898b6a2cf6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:06.281830 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8183d-ed6e-4f60-a87a-898b6a2cf6d4 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:06.281996 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c691b79a-b3aa-4c93-a841-65da39c2d833 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:06.282085 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8183d-ed6e-4f60-a87a-898b6a2cf6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:06.282375 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8183d-ed6e-4f60-a87a-898b6a2cf6d4 None admin] PATH_INFO: `/v3/users/dce468546a1841c0bc2f6bda45393ccd` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:06.283514 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c691b79a-b3aa-4c93-a841-65da39c2d833 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:06.283652 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef8183d-ed6e-4f60-a87a-898b6a2cf6d4 None admin] RBAC: Authorizing `identity:get_user(user_id=dce468546a1841c0bc2f6bda45393ccd)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:06.285118 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef8183d-ed6e-4f60-a87a-898b6a2cf6d4 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:06.285241 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 43/82] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:43:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:06.285872 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c691b79a-b3aa-4c93-a841-65da39c2d833 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:06.291477 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 40/83] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:43:06 2022] GET /identity/v3/users/dce468546a1841c0bc2f6bda45393ccd => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:06.291477 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8183d-ed6e-4f60-a87a-898b6a2cf6d4 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:06.367680 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:06.384498 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c691b79a-b3aa-4c93-a841-65da39c2d833 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:06.386295 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8183d-ed6e-4f60-a87a-898b6a2cf6d4 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:06.418016 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe884bc5-f723-4dce-bcc3-237957ed13eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:06.418870 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe884bc5-f723-4dce-bcc3-237957ed13eb None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:06.419152 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe884bc5-f723-4dce-bcc3-237957ed13eb None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:06.419457 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe884bc5-f723-4dce-bcc3-237957ed13eb None admin] PATH_INFO: `/v3/projects/e67bbf3bcff7420690a69b58a6241244` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:06.419566 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5f216-7cda-4abf-9862-ef082912649f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:06.420164 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5f216-7cda-4abf-9862-ef082912649f None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:06.420453 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5f216-7cda-4abf-9862-ef082912649f None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:06.420627 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe884bc5-f723-4dce-bcc3-237957ed13eb None admin] RBAC: Authorizing `identity:get_project(project_id=e67bbf3bcff7420690a69b58a6241244)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:06.420730 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5f216-7cda-4abf-9862-ef082912649f None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:06.421483 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c5f216-7cda-4abf-9862-ef082912649f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:06.421791 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c5f216-7cda-4abf-9862-ef082912649f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:06.422077 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe884bc5-f723-4dce-bcc3-237957ed13eb None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:06.423162 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c5f216-7cda-4abf-9862-ef082912649f None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:06.423895 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 44/84] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:43:06 2022] GET /identity/v3/projects/e67bbf3bcff7420690a69b58a6241244 => generated 298 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:06.424651 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe884bc5-f723-4dce-bcc3-237957ed13eb None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:06.429145 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:06.432636 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 41/85] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:43:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:06.433486 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5f216-7cda-4abf-9862-ef082912649f None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:06.523023 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe884bc5-f723-4dce-bcc3-237957ed13eb None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:06.531085 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5f216-7cda-4abf-9862-ef082912649f None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:06.556381 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad8cc20-168f-495e-bf2d-b4ec23467c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:06.557015 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8cc20-168f-495e-bf2d-b4ec23467c9b None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:06.557301 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8cc20-168f-495e-bf2d-b4ec23467c9b None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:06.557579 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8cc20-168f-495e-bf2d-b4ec23467c9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:06.564257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f22ca1-7092-4894-b2e3-bc4a3a1be3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:06.564917 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f22ca1-7092-4894-b2e3-bc4a3a1be3e6 None admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:06.565199 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f22ca1-7092-4894-b2e3-bc4a3a1be3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:06.565477 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f22ca1-7092-4894-b2e3-bc4a3a1be3e6 None admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:06.566188 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f22ca1-7092-4894-b2e3-bc4a3a1be3e6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:06.567668 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f22ca1-7092-4894-b2e3-bc4a3a1be3e6 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:06.589954 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:06.595500 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 42/86] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Jun 23 01:43:06 2022] POST /identity/v3/users => generated 289 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:43:06.596133 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f22ca1-7092-4894-b2e3-bc4a3a1be3e6 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:06.626715 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad8cc20-168f-495e-bf2d-b4ec23467c9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:06.628139 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad8cc20-168f-495e-bf2d-b4ec23467c9b None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:06.631324 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:06.707640 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f22ca1-7092-4894-b2e3-bc4a3a1be3e6 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:06.741216 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 45/87] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:43:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 01:43:06.741216 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad8cc20-168f-495e-bf2d-b4ec23467c9b None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:06.754106 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d319ede-9cae-4c35-8e69-5ac951e4917d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:06.754778 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d319ede-9cae-4c35-8e69-5ac951e4917d None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:06.755063 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d319ede-9cae-4c35-8e69-5ac951e4917d None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:06.755429 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d319ede-9cae-4c35-8e69-5ac951e4917d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:06.833425 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:06.838428 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d319ede-9cae-4c35-8e69-5ac951e4917d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:06.840131 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d319ede-9cae-4c35-8e69-5ac951e4917d None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:06.866523 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad8cc20-168f-495e-bf2d-b4ec23467c9b None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:06.869635 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:06.907596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba23e9f-7331-4fad-86a7-d08a45f6d67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:06.908311 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba23e9f-7331-4fad-86a7-d08a45f6d67e None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:06.908602 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba23e9f-7331-4fad-86a7-d08a45f6d67e None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:06.909023 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba23e9f-7331-4fad-86a7-d08a45f6d67e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:06.909879 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba23e9f-7331-4fad-86a7-d08a45f6d67e None admin] RBAC: Adding query filter params (role.id=9021e3c4e3384a309951be4cc1d852e5, scope.project.id=e67bbf3bcff7420690a69b58a6241244, user.id=dce468546a1841c0bc2f6bda45393ccd) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:06.910518 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba23e9f-7331-4fad-86a7-d08a45f6d67e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:06.912097 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba23e9f-7331-4fad-86a7-d08a45f6d67e None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:06.920406 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 46/88] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:43:06 2022] GET /identity/v3/role_assignments?user.id=dce468546a1841c0bc2f6bda45393ccd&scope.project.id=e67bbf3bcff7420690a69b58a6241244&role.id=9021e3c4e3384a309951be4cc1d852e5 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:06.921055 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba23e9f-7331-4fad-86a7-d08a45f6d67e None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:06.967840 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 43/89] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:43:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 01:43:06.968725 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d319ede-9cae-4c35-8e69-5ac951e4917d None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:07.040715 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba23e9f-7331-4fad-86a7-d08a45f6d67e None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:07.082474 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186de256-6632-429f-960b-a939f9aca200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:07.083393 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186de256-6632-429f-960b-a939f9aca200 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:07.083679 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186de256-6632-429f-960b-a939f9aca200 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:07.083991 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186de256-6632-429f-960b-a939f9aca200 None admin] PATH_INFO: `/v3/users/dce468546a1841c0bc2f6bda45393ccd` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:07.085246 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186de256-6632-429f-960b-a939f9aca200 None admin] RBAC: Authorizing `identity:get_user(user_id=dce468546a1841c0bc2f6bda45393ccd)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:07.086741 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186de256-6632-429f-960b-a939f9aca200 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:07.088803 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 47/90] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:43:06 2022] GET /identity/v3/users/dce468546a1841c0bc2f6bda45393ccd => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:07.089603 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186de256-6632-429f-960b-a939f9aca200 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:07.103879 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:07.108598 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d319ede-9cae-4c35-8e69-5ac951e4917d None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:07.127468 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:07.163862 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd4e998-80d9-4e1a-9fed-a31226814a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:07.164760 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd4e998-80d9-4e1a-9fed-a31226814a88 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:07.165046 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd4e998-80d9-4e1a-9fed-a31226814a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:07.165325 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd4e998-80d9-4e1a-9fed-a31226814a88 None admin] PATH_INFO: `/v3/projects/e67bbf3bcff7420690a69b58a6241244` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:07.170199 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd4e998-80d9-4e1a-9fed-a31226814a88 None admin] RBAC: Authorizing `identity:get_project(project_id=e67bbf3bcff7420690a69b58a6241244)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:07.171931 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd4e998-80d9-4e1a-9fed-a31226814a88 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:07.173845 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 44/91] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:43:06 2022] GET /identity/v3/projects/e67bbf3bcff7420690a69b58a6241244 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:07.174634 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd4e998-80d9-4e1a-9fed-a31226814a88 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:07.196982 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186de256-6632-429f-960b-a939f9aca200 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:07.243920 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d799375f-a3eb-40f2-9f94-9b1bd3f47b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:07.244794 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d799375f-a3eb-40f2-9f94-9b1bd3f47b60 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:07.245071 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d799375f-a3eb-40f2-9f94-9b1bd3f47b60 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:07.245404 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d799375f-a3eb-40f2-9f94-9b1bd3f47b60 None admin] PATH_INFO: `/v3/projects/015b322c1b854846b17c9f0470487f18` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:07.246837 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d799375f-a3eb-40f2-9f94-9b1bd3f47b60 None admin] RBAC: Authorizing `identity:get_project(project_id=015b322c1b854846b17c9f0470487f18)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:07.248418 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d799375f-a3eb-40f2-9f94-9b1bd3f47b60 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:07.250412 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 48/92] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:43:07 2022] GET /identity/v3/projects/015b322c1b854846b17c9f0470487f18 => generated 312 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:07.251080 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d799375f-a3eb-40f2-9f94-9b1bd3f47b60 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:07.310465 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd4e998-80d9-4e1a-9fed-a31226814a88 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:07.345898 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e8dbd-a085-4910-bfb2-10cc1359849a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:07.346556 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e8dbd-a085-4910-bfb2-10cc1359849a None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:07.346836 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e8dbd-a085-4910-bfb2-10cc1359849a None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:07.347110 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e8dbd-a085-4910-bfb2-10cc1359849a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:07.378303 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d799375f-a3eb-40f2-9f94-9b1bd3f47b60 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:07.382133 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:07.397410 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:07.430717 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42279c80-b0d9-4c1e-8978-b625a849d8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:07.433290 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42279c80-b0d9-4c1e-8978-b625a849d8b2 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:07.433574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42279c80-b0d9-4c1e-8978-b625a849d8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:07.433849 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42279c80-b0d9-4c1e-8978-b625a849d8b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:07.434702 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42279c80-b0d9-4c1e-8978-b625a849d8b2 None admin] RBAC: Adding query filter params (role.id=7b20d65f49ee4a48894b6c060bb5d580, scope.project.id=e67bbf3bcff7420690a69b58a6241244, user.id=34e5aabcf8f5425f9c9f312f635ff77a) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:07.435013 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42279c80-b0d9-4c1e-8978-b625a849d8b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:07.440589 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42279c80-b0d9-4c1e-8978-b625a849d8b2 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:07.450702 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49e8dbd-a085-4910-bfb2-10cc1359849a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:07.452200 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49e8dbd-a085-4910-bfb2-10cc1359849a None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:07.455592 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 49/93] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:43:07 2022] GET /identity/v3/role_assignments?user.id=34e5aabcf8f5425f9c9f312f635ff77a&scope.project.id=e67bbf3bcff7420690a69b58a6241244&role.id=7b20d65f49ee4a48894b6c060bb5d580 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:07.456375 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42279c80-b0d9-4c1e-8978-b625a849d8b2 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:07.598980 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 45/94] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:43:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 01:43:07.604762 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e8dbd-a085-4910-bfb2-10cc1359849a None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:07.624123 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42279c80-b0d9-4c1e-8978-b625a849d8b2 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:07.661369 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3939b5-a786-4bc4-ac68-197aed0d2f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:07.662221 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3939b5-a786-4bc4-ac68-197aed0d2f50 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:07.662503 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3939b5-a786-4bc4-ac68-197aed0d2f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:07.662777 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3939b5-a786-4bc4-ac68-197aed0d2f50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:07.663660 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3939b5-a786-4bc4-ac68-197aed0d2f50 None admin] RBAC: Adding query filter params (role.id=9021e3c4e3384a309951be4cc1d852e5, scope.project.id=015b322c1b854846b17c9f0470487f18, user.id=dce468546a1841c0bc2f6bda45393ccd) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:07.663975 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3939b5-a786-4bc4-ac68-197aed0d2f50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:07.665640 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3939b5-a786-4bc4-ac68-197aed0d2f50 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:07.672280 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 50/95] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:43:07 2022] GET /identity/v3/role_assignments?user.id=dce468546a1841c0bc2f6bda45393ccd&scope.project.id=015b322c1b854846b17c9f0470487f18&role.id=9021e3c4e3384a309951be4cc1d852e5 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:07.674917 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:07.737492 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e8dbd-a085-4910-bfb2-10cc1359849a None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:07.774490 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e25618-1343-4ac8-9468-dc88a26cb9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:07.775443 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e25618-1343-4ac8-9468-dc88a26cb9bd None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:07.775723 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e25618-1343-4ac8-9468-dc88a26cb9bd None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:07.776000 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e25618-1343-4ac8-9468-dc88a26cb9bd None admin] PATH_INFO: `/v3/projects/e67bbf3bcff7420690a69b58a6241244` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:07.777388 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e25618-1343-4ac8-9468-dc88a26cb9bd None admin] RBAC: Authorizing `identity:get_project(project_id=e67bbf3bcff7420690a69b58a6241244)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:07.778878 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e25618-1343-4ac8-9468-dc88a26cb9bd None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:07.781183 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 46/96] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:43:07 2022] GET /identity/v3/projects/e67bbf3bcff7420690a69b58a6241244 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:43:07.788404 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3939b5-a786-4bc4-ac68-197aed0d2f50 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:07.823645 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:07.897020 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3939b5-a786-4bc4-ac68-197aed0d2f50 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:07.933012 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d140a49-758f-4dbd-9c6d-5f0972ac5122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:07.933670 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d140a49-758f-4dbd-9c6d-5f0972ac5122 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:07.933949 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d140a49-758f-4dbd-9c6d-5f0972ac5122 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:07.934226 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d140a49-758f-4dbd-9c6d-5f0972ac5122 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:07.935081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d140a49-758f-4dbd-9c6d-5f0972ac5122 None admin] RBAC: Adding query filter params (role.id=1c7adeca27524e258f7ddd2c107e43e1, scope.project.id=e67bbf3bcff7420690a69b58a6241244, user.id=dce468546a1841c0bc2f6bda45393ccd) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:07.935429 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d140a49-758f-4dbd-9c6d-5f0972ac5122 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:07.936911 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d140a49-758f-4dbd-9c6d-5f0972ac5122 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:07.942933 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 51/97] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:43:07 2022] GET /identity/v3/role_assignments?user.id=dce468546a1841c0bc2f6bda45393ccd&scope.project.id=e67bbf3bcff7420690a69b58a6241244&role.id=1c7adeca27524e258f7ddd2c107e43e1 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:09.763929 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792ea03-164d-4235-ab44-036766af3c0b None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:09.764952 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792ea03-164d-4235-ab44-036766af3c0b None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:09.765427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792ea03-164d-4235-ab44-036766af3c0b None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:09.767302 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 47/98] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:09.773072 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef950b9-3b7e-4814-b276-536c307584ae None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:09.773876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef950b9-3b7e-4814-b276-536c307584ae None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:09.774360 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef950b9-3b7e-4814-b276-536c307584ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:09.797132 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-0ef950b9-3b7e-4814-b276-536c307584ae None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:09.798633 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-0ef950b9-3b7e-4814-b276-536c307584ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:09.842885 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad3668a-1d57-4fc5-bca8-e0d7b8c92789 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:09.843538 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad3668a-1d57-4fc5-bca8-e0d7b8c92789 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:09.844040 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad3668a-1d57-4fc5-bca8-e0d7b8c92789 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:09.845618 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 48/99] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:09.850653 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ce8f9-3312-4c2c-89a5-563b473ef338 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:09.851170 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ce8f9-3312-4c2c-89a5-563b473ef338 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:09.851712 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ce8f9-3312-4c2c-89a5-563b473ef338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:09.877191 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-5d8ce8f9-3312-4c2c-89a5-563b473ef338 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:09.879969 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-5d8ce8f9-3312-4c2c-89a5-563b473ef338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:09.912770 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 52/100] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:09.952546 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef950b9-3b7e-4814-b276-536c307584ae None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:09.988316 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 49/101] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 01:43:09.997912 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:10.033271 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ce8f9-3312-4c2c-89a5-563b473ef338 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:10.110445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef950b9-3b7e-4814-b276-536c307584ae None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:10.149354 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:10.181499 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e6f557-021b-41e8-992c-ad20589a59fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:10.182791 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6f557-021b-41e8-992c-ad20589a59fe None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:10.183338 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6f557-021b-41e8-992c-ad20589a59fe None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:10.183810 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6f557-021b-41e8-992c-ad20589a59fe None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:10.197085 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e6f557-021b-41e8-992c-ad20589a59fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:10.206151 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-f1e6f557-021b-41e8-992c-ad20589a59fe None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 01:43:10.206151 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:10.206151 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:10.206151 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:10.206151 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:10.206151 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:10.206151 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:10.206151 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:10.206151 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:10.206151 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:10.206151 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:10.206151 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:10.206151 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:10.206151 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:10.206151 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:10.206151 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:10.207003 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:10.207003 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:10.207003 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:10.207003 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:10.207003 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:10.207003 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:10.207003 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:10.207003 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:10.207003 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:10.207003 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:10.207003 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:10.207003 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:10.207003 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:10.207003 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:10.207003 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:10.207003 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:10.207776 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:10.207776 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:10.207776 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:10.207776 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:10.207776 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:10.207776 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:10.207776 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:10.207776 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:10.207776 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:10.207776 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:10.207776 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 01:43:10.207776 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:43:10.210701 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 53/102] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:43:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:43:10.212287 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216eb66-86cd-43dc-9e1b-687db560ac3d None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:10.212772 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216eb66-86cd-43dc-9e1b-687db560ac3d None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:10.213254 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216eb66-86cd-43dc-9e1b-687db560ac3d None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:10.214932 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 54/103] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:10.223464 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ce8f9-3312-4c2c-89a5-563b473ef338 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:10.224601 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9216eb66-86cd-43dc-9e1b-687db560ac3d None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:10.262766 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c21f834-a2c4-41bd-aae5-8bf71a987864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:10.263621 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21f834-a2c4-41bd-aae5-8bf71a987864 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:10.263901 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21f834-a2c4-41bd-aae5-8bf71a987864 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:10.264174 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21f834-a2c4-41bd-aae5-8bf71a987864 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:10.265371 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c21f834-a2c4-41bd-aae5-8bf71a987864 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:10.266955 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c21f834-a2c4-41bd-aae5-8bf71a987864 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:10.268688 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 50/104] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:43:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:10.270008 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4223cd8-4fd7-419f-a38d-82d8d57db6e3 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:10.270293 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4223cd8-4fd7-419f-a38d-82d8d57db6e3 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:10.270572 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4223cd8-4fd7-419f-a38d-82d8d57db6e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:10.287919 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-b4223cd8-4fd7-419f-a38d-82d8d57db6e3 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:10.289029 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-b4223cd8-4fd7-419f-a38d-82d8d57db6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:10.346928 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:10.366785 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9216eb66-86cd-43dc-9e1b-687db560ac3d None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:10.382203 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 51/105] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:10.384505 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9270c5-cdaa-42e8-9b1b-8d1a1460b24e None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:10.384986 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9270c5-cdaa-42e8-9b1b-8d1a1460b24e None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:10.385719 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9270c5-cdaa-42e8-9b1b-8d1a1460b24e None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:10.387582 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 52/106] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:10.388444 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9270c5-cdaa-42e8-9b1b-8d1a1460b24e None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:10.408458 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284bc3c9-650c-46c6-92ac-a41d62cebbc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:10.409153 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284bc3c9-650c-46c6-92ac-a41d62cebbc0 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:10.409508 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284bc3c9-650c-46c6-92ac-a41d62cebbc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:10.409789 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284bc3c9-650c-46c6-92ac-a41d62cebbc0 None admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:10.410554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284bc3c9-650c-46c6-92ac-a41d62cebbc0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:10.411061 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284bc3c9-650c-46c6-92ac-a41d62cebbc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:10.412535 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284bc3c9-650c-46c6-92ac-a41d62cebbc0 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:10.422657 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 55/107] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jun 23 01:43:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:10.428480 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a552b-b88d-4216-be29-dca35a25679d None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:10.428759 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a552b-b88d-4216-be29-dca35a25679d None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:10.429037 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a552b-b88d-4216-be29-dca35a25679d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:10.448397 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-353a552b-b88d-4216-be29-dca35a25679d None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:10.449790 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-353a552b-b88d-4216-be29-dca35a25679d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:10.458131 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:10.498826 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9270c5-cdaa-42e8-9b1b-8d1a1460b24e None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:10.524828 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 56/108] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:10.532193 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353a552b-b88d-4216-be29-dca35a25679d None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:10.534160 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539f3346-457e-4b38-bb71-1d6111b8bb2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:10.535067 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f3346-457e-4b38-bb71-1d6111b8bb2e None admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:10.535557 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f3346-457e-4b38-bb71-1d6111b8bb2e None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:10.536015 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f3346-457e-4b38-bb71-1d6111b8bb2e None admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:10.537030 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539f3346-457e-4b38-bb71-1d6111b8bb2e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:10.538777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539f3346-457e-4b38-bb71-1d6111b8bb2e None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:10.567604 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 53/109] 10.222.0.26 () {60 vars in 1141 bytes} [Thu Jun 23 01:43:10 2022] POST /identity/v3/projects => generated 302 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:43:10.568964 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539f3346-457e-4b38-bb71-1d6111b8bb2e None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:10.647241 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:10.672317 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353a552b-b88d-4216-be29-dca35a25679d None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:10.701331 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539f3346-457e-4b38-bb71-1d6111b8bb2e None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:10.717713 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:10.722025 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38530ce7-ddb8-417f-8d1c-4b3da940aa49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:10.724100 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38530ce7-ddb8-417f-8d1c-4b3da940aa49 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:10.724621 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38530ce7-ddb8-417f-8d1c-4b3da940aa49 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:10.725064 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38530ce7-ddb8-417f-8d1c-4b3da940aa49 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:10.743773 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38530ce7-ddb8-417f-8d1c-4b3da940aa49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:10.751856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-38530ce7-ddb8-417f-8d1c-4b3da940aa49 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 01:43:10.751856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:10.751856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:10.751856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:10.751856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:10.751856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:10.751856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:10.751856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:10.751856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:10.751856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:10.751856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:10.751856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:10.751856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:10.751856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:10.751856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:10.751856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:10.752683 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:10.752683 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:10.752683 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:10.752683 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:10.752683 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:10.752683 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:10.752683 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:10.752683 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:10.752683 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:10.752683 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:10.752683 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:10.752683 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:10.752683 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:10.752683 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:10.752683 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:10.752683 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:10.753854 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:10.753854 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:10.753854 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:10.753854 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:10.753854 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:10.753854 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:10.753854 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:10.753854 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:10.753854 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:10.753854 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:10.753854 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 01:43:10.753854 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:43:10.753854 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 57/110] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:43:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:43:10.754471 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38530ce7-ddb8-417f-8d1c-4b3da940aa49 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:10.755572 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d076b50b-488e-476e-8a62-185fcba6e6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:10.756720 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076b50b-488e-476e-8a62-185fcba6e6fb None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:10.757186 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076b50b-488e-476e-8a62-185fcba6e6fb None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:10.757760 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076b50b-488e-476e-8a62-185fcba6e6fb None admin] PATH_INFO: `/v3/users/9df85dac31484bdbaa8f0702a4d52e5a` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:10.777280 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d076b50b-488e-476e-8a62-185fcba6e6fb None admin] RBAC: Authorizing `identity:get_user(user_id=9df85dac31484bdbaa8f0702a4d52e5a)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:10.779170 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d076b50b-488e-476e-8a62-185fcba6e6fb None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:10.781813 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 54/111] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:43:10 2022] GET /identity/v3/users/9df85dac31484bdbaa8f0702a4d52e5a => generated 289 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:10.783417 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e02ec-9eee-4eff-9aef-a574dbe3a7f3 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:10.783957 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e02ec-9eee-4eff-9aef-a574dbe3a7f3 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:10.784242 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e02ec-9eee-4eff-9aef-a574dbe3a7f3 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:10.786209 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 55/112] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:10.787322 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e02ec-9eee-4eff-9aef-a574dbe3a7f3 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:10.867183 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38530ce7-ddb8-417f-8d1c-4b3da940aa49 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:10.900938 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e02ec-9eee-4eff-9aef-a574dbe3a7f3 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:10.902718 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27900922-5618-4b83-9f00-d5d64e769c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:10.903559 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27900922-5618-4b83-9f00-d5d64e769c48 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:10.903847 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27900922-5618-4b83-9f00-d5d64e769c48 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:10.904124 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27900922-5618-4b83-9f00-d5d64e769c48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:10.911763 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:10.917973 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27900922-5618-4b83-9f00-d5d64e769c48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:10.928608 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-27900922-5618-4b83-9f00-d5d64e769c48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:43:10.928608 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:10.928608 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:10.928608 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:10.928608 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:10.928608 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:10.928608 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:10.928608 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:10.928608 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:10.928608 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:10.928608 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:10.928608 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:10.928608 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:10.928608 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:10.928608 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:10.928608 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:10.929541 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:10.929541 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:10.929541 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:10.929541 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:10.929541 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:10.929541 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:10.929541 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:10.929541 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:10.929541 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:10.929541 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:10.929541 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:10.929541 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:10.929541 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:10.929541 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:10.929541 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:10.929541 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:10.930390 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:10.930390 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:10.930390 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:10.930390 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:10.930390 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:10.930390 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:10.930390 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:10.930390 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:10.930390 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:10.930390 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:10.930390 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:43:10.930390 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:43:10.932714 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 58/113] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jun 23 01:43:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:43:10.937578 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27900922-5618-4b83-9f00-d5d64e769c48 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:10.940544 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81acedcf-159d-4924-ba35-d966d88ec075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:10.941243 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81acedcf-159d-4924-ba35-d966d88ec075 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:10.941525 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81acedcf-159d-4924-ba35-d966d88ec075 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:10.941800 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81acedcf-159d-4924-ba35-d966d88ec075 None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:10.942556 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81acedcf-159d-4924-ba35-d966d88ec075 None admin] RBAC: Adding query filter params (name=member) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:10.942864 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81acedcf-159d-4924-ba35-d966d88ec075 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:10.944369 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81acedcf-159d-4924-ba35-d966d88ec075 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:10.953373 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 56/114] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jun 23 01:43:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:10.954426 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a802b3c-7e41-456f-b9ab-414b3075dbe7 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:10.954745 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a802b3c-7e41-456f-b9ab-414b3075dbe7 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:10.954992 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a802b3c-7e41-456f-b9ab-414b3075dbe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:10.979180 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-4a802b3c-7e41-456f-b9ab-414b3075dbe7 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:10.980431 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-4a802b3c-7e41-456f-b9ab-414b3075dbe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:11.008658 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:11.046962 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27900922-5618-4b83-9f00-d5d64e769c48 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:11.049742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 57/115] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:11.059387 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2350ad-29a5-48a5-9b7b-4b980a7e9819 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:11.059387 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2350ad-29a5-48a5-9b7b-4b980a7e9819 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:11.059387 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2350ad-29a5-48a5-9b7b-4b980a7e9819 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:11.059387 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 58/116] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:11.059387 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2350ad-29a5-48a5-9b7b-4b980a7e9819 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:11.102292 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcebe4d-9e8a-4e1c-b294-1a1f5a10d4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:11.102957 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcebe4d-9e8a-4e1c-b294-1a1f5a10d4af None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:11.103260 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcebe4d-9e8a-4e1c-b294-1a1f5a10d4af None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:11.103542 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcebe4d-9e8a-4e1c-b294-1a1f5a10d4af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:11.159030 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2350ad-29a5-48a5-9b7b-4b980a7e9819 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:11.170552 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:11.174718 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcebe4d-9e8a-4e1c-b294-1a1f5a10d4af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:11.177057 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcebe4d-9e8a-4e1c-b294-1a1f5a10d4af None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:11.197485 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3107f012-6f60-46da-92f8-ad4919ef2d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:11.198250 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107f012-6f60-46da-92f8-ad4919ef2d5f None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:11.198604 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107f012-6f60-46da-92f8-ad4919ef2d5f None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:11.198951 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107f012-6f60-46da-92f8-ad4919ef2d5f None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:11.200215 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3107f012-6f60-46da-92f8-ad4919ef2d5f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:11.200603 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3107f012-6f60-46da-92f8-ad4919ef2d5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:11.202068 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3107f012-6f60-46da-92f8-ad4919ef2d5f None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:11.215553 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 59/117] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:43:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:11.215553 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3107f012-6f60-46da-92f8-ad4919ef2d5f None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:11.245788 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:11.287712 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 59/118] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:43:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 01:43:11.289038 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b70c3a-20ac-45d0-80ff-de868ad13747 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:11.289380 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b70c3a-20ac-45d0-80ff-de868ad13747 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:11.292238 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b70c3a-20ac-45d0-80ff-de868ad13747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:11.308785 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-17b70c3a-20ac-45d0-80ff-de868ad13747 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:11.309978 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-17b70c3a-20ac-45d0-80ff-de868ad13747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:11.331246 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3107f012-6f60-46da-92f8-ad4919ef2d5f None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:11.367063 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dcb547-a6bf-464c-82ce-7efd4baba8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:11.368117 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dcb547-a6bf-464c-82ce-7efd4baba8cc None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:11.368579 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dcb547-a6bf-464c-82ce-7efd4baba8cc None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:11.368747 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dcb547-a6bf-464c-82ce-7efd4baba8cc None admin] PATH_INFO: `/v3/users/dce468546a1841c0bc2f6bda45393ccd` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:11.369968 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dcb547-a6bf-464c-82ce-7efd4baba8cc None admin] RBAC: Authorizing `identity:get_user(user_id=dce468546a1841c0bc2f6bda45393ccd)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:11.371460 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dcb547-a6bf-464c-82ce-7efd4baba8cc None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:11.373297 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 60/119] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:43:11 2022] GET /identity/v3/users/dce468546a1841c0bc2f6bda45393ccd => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:11.378272 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dcb547-a6bf-464c-82ce-7efd4baba8cc None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:11.387231 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 60/120] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:11.387231 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b70c3a-20ac-45d0-80ff-de868ad13747 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:11.415340 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:11.491372 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:11.497053 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b70c3a-20ac-45d0-80ff-de868ad13747 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:11.498531 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dcb547-a6bf-464c-82ce-7efd4baba8cc None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:11.532563 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e3b48-7fd4-4b81-a70e-afff14310a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:11.533441 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e3b48-7fd4-4b81-a70e-afff14310a3c None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:11.533726 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e3b48-7fd4-4b81-a70e-afff14310a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:11.534005 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e3b48-7fd4-4b81-a70e-afff14310a3c None admin] PATH_INFO: `/v3/users/34e5aabcf8f5425f9c9f312f635ff77a` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:11.535205 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926e3b48-7fd4-4b81-a70e-afff14310a3c None admin] RBAC: Authorizing `identity:get_user(user_id=34e5aabcf8f5425f9c9f312f635ff77a)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:11.536371 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf208269-0218-427a-97e4-049c702ed5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:11.536685 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926e3b48-7fd4-4b81-a70e-afff14310a3c None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:11.537345 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf208269-0218-427a-97e4-049c702ed5d2 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:11.537694 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf208269-0218-427a-97e4-049c702ed5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:11.538033 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf208269-0218-427a-97e4-049c702ed5d2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:11.542963 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 61/121] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:43:11 2022] GET /identity/v3/users/34e5aabcf8f5425f9c9f312f635ff77a => generated 247 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:11.542963 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e3b48-7fd4-4b81-a70e-afff14310a3c None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:11.553211 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf208269-0218-427a-97e4-049c702ed5d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:11.561318 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-cf208269-0218-427a-97e4-049c702ed5d2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 01:43:11.561318 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:11.561318 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:11.561318 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:11.561318 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:11.561318 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:11.561318 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:11.561318 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:11.561318 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:11.561318 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:11.561318 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:11.561318 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:11.561318 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:11.561318 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:11.561318 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:11.561318 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:11.562202 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:11.562202 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:11.562202 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:11.562202 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:11.562202 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:11.562202 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:11.562202 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:11.562202 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:11.562202 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:11.562202 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:11.562202 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:11.562202 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:11.562202 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:11.562202 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:11.562202 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:11.562202 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:11.562985 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:11.562985 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:11.562985 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:11.562985 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:11.562985 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:11.562985 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:11.562985 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:11.562985 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:11.562985 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:11.562985 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:11.562985 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 01:43:11.562985 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:43:11.562985 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 61/122] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:43:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:43:11.568191 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf208269-0218-427a-97e4-049c702ed5d2 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:11.648594 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:11.655306 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e3b48-7fd4-4b81-a70e-afff14310a3c None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:11.674634 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf208269-0218-427a-97e4-049c702ed5d2 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:11.689254 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c846400-f992-4cc5-8fbd-6bc0b87aa106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:11.690120 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c846400-f992-4cc5-8fbd-6bc0b87aa106 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:11.690408 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c846400-f992-4cc5-8fbd-6bc0b87aa106 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:11.690692 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c846400-f992-4cc5-8fbd-6bc0b87aa106 None admin] PATH_INFO: `/v3/projects/e67bbf3bcff7420690a69b58a6241244` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:11.691927 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c846400-f992-4cc5-8fbd-6bc0b87aa106 None admin] RBAC: Authorizing `identity:get_project(project_id=e67bbf3bcff7420690a69b58a6241244)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:11.693422 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c846400-f992-4cc5-8fbd-6bc0b87aa106 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:11.695466 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 62/123] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:43:11 2022] GET /identity/v3/projects/e67bbf3bcff7420690a69b58a6241244 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:11.696237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c846400-f992-4cc5-8fbd-6bc0b87aa106 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:11.713499 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63826f3d-a9b6-4948-9ffc-1424f52581d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:11.714196 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63826f3d-a9b6-4948-9ffc-1424f52581d9 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:11.714486 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63826f3d-a9b6-4948-9ffc-1424f52581d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:11.714768 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63826f3d-a9b6-4948-9ffc-1424f52581d9 None admin] PATH_INFO: `/v3/projects/e67bbf3bcff7420690a69b58a6241244` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:11.715989 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63826f3d-a9b6-4948-9ffc-1424f52581d9 None admin] RBAC: Authorizing `identity:get_project(project_id=e67bbf3bcff7420690a69b58a6241244)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:11.717461 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63826f3d-a9b6-4948-9ffc-1424f52581d9 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:11.723786 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 62/124] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:43:11 2022] GET /identity/v3/projects/e67bbf3bcff7420690a69b58a6241244 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:11.723786 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63826f3d-a9b6-4948-9ffc-1424f52581d9 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:11.747607 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:11.812326 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c846400-f992-4cc5-8fbd-6bc0b87aa106 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:11.835922 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63826f3d-a9b6-4948-9ffc-1424f52581d9 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:11.851441 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7426658d-2e81-4faf-bf8e-08005d407dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:11.852297 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7426658d-2e81-4faf-bf8e-08005d407dd7 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:11.852591 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7426658d-2e81-4faf-bf8e-08005d407dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:11.852875 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7426658d-2e81-4faf-bf8e-08005d407dd7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:11.863157 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7426658d-2e81-4faf-bf8e-08005d407dd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:11.871505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-7426658d-2e81-4faf-bf8e-08005d407dd7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 01:43:11.871505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:11.871505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:11.871505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:11.871505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:11.871505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:11.871505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:11.871505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:11.871505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:11.871505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:11.871505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:11.871505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:11.871505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:11.871505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:11.871505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:11.871505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:11.872895 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:11.872895 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:11.872895 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:11.872895 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:11.872895 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:11.872895 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:11.872895 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:11.872895 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:11.872895 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:11.872895 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:11.872895 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:11.872895 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:11.872895 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:11.872895 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:11.872895 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:11.872895 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:11.873652 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:11.873652 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:11.873652 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:11.873652 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:11.873652 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:11.873652 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:11.873652 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:11.873652 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:11.873652 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:11.873652 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:11.873652 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 01:43:11.873652 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:43:11.873652 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 63/125] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Jun 23 01:43:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:43:11.873652 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7426658d-2e81-4faf-bf8e-08005d407dd7 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:11.877634 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea32a06d-cc65-4a79-b30a-dc721c581acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:11.878434 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea32a06d-cc65-4a79-b30a-dc721c581acb None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:11.878719 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea32a06d-cc65-4a79-b30a-dc721c581acb None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:11.878999 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea32a06d-cc65-4a79-b30a-dc721c581acb None admin] PATH_INFO: `/v3/projects/e67bbf3bcff7420690a69b58a6241244` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:11.880237 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea32a06d-cc65-4a79-b30a-dc721c581acb None admin] RBAC: Authorizing `identity:get_project(project_id=e67bbf3bcff7420690a69b58a6241244)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:11.881743 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea32a06d-cc65-4a79-b30a-dc721c581acb None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:11.888125 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 63/126] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:43:11 2022] GET /identity/v3/projects/e67bbf3bcff7420690a69b58a6241244 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:11.888125 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea32a06d-cc65-4a79-b30a-dc721c581acb None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:11.904646 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:11.980814 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7426658d-2e81-4faf-bf8e-08005d407dd7 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:11.991783 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:12.003247 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea32a06d-cc65-4a79-b30a-dc721c581acb None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:12.018836 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2915d39-8579-4d8d-ab50-69a9ba7e37f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:12.019541 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2915d39-8579-4d8d-ab50-69a9ba7e37f2 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:12.019846 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2915d39-8579-4d8d-ab50-69a9ba7e37f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:12.020105 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2915d39-8579-4d8d-ab50-69a9ba7e37f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:12.020866 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2915d39-8579-4d8d-ab50-69a9ba7e37f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:12.021210 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2915d39-8579-4d8d-ab50-69a9ba7e37f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:12.022651 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2915d39-8579-4d8d-ab50-69a9ba7e37f2 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:12.030905 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 64/127] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jun 23 01:43:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:12.035540 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2915d39-8579-4d8d-ab50-69a9ba7e37f2 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:12.043016 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6bd0a2-b610-47e5-8f3b-e31b94b252b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:12.043930 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6bd0a2-b610-47e5-8f3b-e31b94b252b5 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:12.044278 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6bd0a2-b610-47e5-8f3b-e31b94b252b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:12.044625 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6bd0a2-b610-47e5-8f3b-e31b94b252b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:12.045665 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6bd0a2-b610-47e5-8f3b-e31b94b252b5 None admin] RBAC: Adding query filter params (role.id=d159e6edeef94e418a4616504e214063, scope.project.id=e67bbf3bcff7420690a69b58a6241244, user.id=9df85dac31484bdbaa8f0702a4d52e5a) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:12.046044 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6bd0a2-b610-47e5-8f3b-e31b94b252b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:12.047736 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6bd0a2-b610-47e5-8f3b-e31b94b252b5 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:12.053867 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 64/128] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:43:11 2022] GET /identity/v3/role_assignments?user.id=9df85dac31484bdbaa8f0702a4d52e5a&scope.project.id=e67bbf3bcff7420690a69b58a6241244&role.id=d159e6edeef94e418a4616504e214063 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:12.064340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6bd0a2-b610-47e5-8f3b-e31b94b252b5 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:12.127790 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:12.146790 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2915d39-8579-4d8d-ab50-69a9ba7e37f2 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:12.185344 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da14738f-be6d-4630-b694-a3d253b961b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:12.186176 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14738f-be6d-4630-b694-a3d253b961b3 None admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:12.186528 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14738f-be6d-4630-b694-a3d253b961b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:12.186881 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14738f-be6d-4630-b694-a3d253b961b3 None admin] PATH_INFO: `/v3/projects/e67bbf3bcff7420690a69b58a6241244/users/dce468546a1841c0bc2f6bda45393ccd/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:12.189166 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da14738f-be6d-4630-b694-a3d253b961b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=e67bbf3bcff7420690a69b58a6241244, user_id=dce468546a1841c0bc2f6bda45393ccd, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:12.190838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da14738f-be6d-4630-b694-a3d253b961b3 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:12.219236 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 65/129] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jun 23 01:43:12 2022] PUT /identity/v3/projects/e67bbf3bcff7420690a69b58a6241244/users/dce468546a1841c0bc2f6bda45393ccd/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:43:12.219457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6bd0a2-b610-47e5-8f3b-e31b94b252b5 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:12.219542 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da14738f-be6d-4630-b694-a3d253b961b3 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:12.256704 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad26108-7939-46a3-b667-832f368df9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:12.257316 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad26108-7939-46a3-b667-832f368df9f5 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:12.257607 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad26108-7939-46a3-b667-832f368df9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:12.257885 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad26108-7939-46a3-b667-832f368df9f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:12.258657 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad26108-7939-46a3-b667-832f368df9f5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:12.258971 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad26108-7939-46a3-b667-832f368df9f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:12.264834 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad26108-7939-46a3-b667-832f368df9f5 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:12.282548 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 65/130] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:43:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:12.283860 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad26108-7939-46a3-b667-832f368df9f5 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:12.322866 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:12.356064 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da14738f-be6d-4630-b694-a3d253b961b3 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:12.403403 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99531ffe-d625-43e5-a9d4-6e9c3496b5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:12.406523 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99531ffe-d625-43e5-a9d4-6e9c3496b5d2 None admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:12.407858 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99531ffe-d625-43e5-a9d4-6e9c3496b5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:12.408307 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99531ffe-d625-43e5-a9d4-6e9c3496b5d2 None admin] PATH_INFO: `/v3/projects/e67bbf3bcff7420690a69b58a6241244/users/34e5aabcf8f5425f9c9f312f635ff77a/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:12.413667 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99531ffe-d625-43e5-a9d4-6e9c3496b5d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=e67bbf3bcff7420690a69b58a6241244, user_id=34e5aabcf8f5425f9c9f312f635ff77a, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:12.416364 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99531ffe-d625-43e5-a9d4-6e9c3496b5d2 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:12.450384 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 66/131] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jun 23 01:43:12 2022] PUT /identity/v3/projects/e67bbf3bcff7420690a69b58a6241244/users/34e5aabcf8f5425f9c9f312f635ff77a/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:43:12.455623 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99531ffe-d625-43e5-a9d4-6e9c3496b5d2 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:12.468561 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:12.504593 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad26108-7939-46a3-b667-832f368df9f5 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:12.566297 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe03f9-24c0-4cf1-b800-5d63548c0eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:12.567087 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe03f9-24c0-4cf1-b800-5d63548c0eb6 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:12.567397 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe03f9-24c0-4cf1-b800-5d63548c0eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:12.567670 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe03f9-24c0-4cf1-b800-5d63548c0eb6 None admin] PATH_INFO: `/v3/users/dce468546a1841c0bc2f6bda45393ccd` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:12.569054 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe03f9-24c0-4cf1-b800-5d63548c0eb6 None admin] RBAC: Authorizing `identity:get_user(user_id=dce468546a1841c0bc2f6bda45393ccd)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:12.570725 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe03f9-24c0-4cf1-b800-5d63548c0eb6 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:12.572561 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 66/132] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:43:12 2022] GET /identity/v3/users/dce468546a1841c0bc2f6bda45393ccd => generated 275 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:12.578376 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe03f9-24c0-4cf1-b800-5d63548c0eb6 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:12.580312 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99531ffe-d625-43e5-a9d4-6e9c3496b5d2 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:12.622048 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda75590-54d8-4437-82c9-fb4173417f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:12.624957 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda75590-54d8-4437-82c9-fb4173417f24 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:12.625537 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda75590-54d8-4437-82c9-fb4173417f24 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:12.626022 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda75590-54d8-4437-82c9-fb4173417f24 None admin] PATH_INFO: `/v3/users/dce468546a1841c0bc2f6bda45393ccd` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:12.628715 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda75590-54d8-4437-82c9-fb4173417f24 None admin] RBAC: Authorizing `identity:get_user(user_id=dce468546a1841c0bc2f6bda45393ccd)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:12.631353 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda75590-54d8-4437-82c9-fb4173417f24 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:12.636438 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 67/133] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:43:12 2022] GET /identity/v3/users/dce468546a1841c0bc2f6bda45393ccd => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:12.643296 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda75590-54d8-4437-82c9-fb4173417f24 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:12.760515 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:12.783309 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe03f9-24c0-4cf1-b800-5d63548c0eb6 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:12.789800 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:12.823775 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda75590-54d8-4437-82c9-fb4173417f24 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:12.828009 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad35caab-6b12-48da-96d1-a55f88881696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:12.828898 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad35caab-6b12-48da-96d1-a55f88881696 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:12.829181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad35caab-6b12-48da-96d1-a55f88881696 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:12.829454 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad35caab-6b12-48da-96d1-a55f88881696 None admin] PATH_INFO: `/v3/projects/015b322c1b854846b17c9f0470487f18` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:12.830655 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad35caab-6b12-48da-96d1-a55f88881696 None admin] RBAC: Authorizing `identity:get_project(project_id=015b322c1b854846b17c9f0470487f18)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:12.832746 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad35caab-6b12-48da-96d1-a55f88881696 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:12.835371 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 67/134] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:43:12 2022] GET /identity/v3/projects/015b322c1b854846b17c9f0470487f18 => generated 312 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:12.837027 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5399281-77d8-4d88-a3c4-095c1a487a56 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:12.837585 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5399281-77d8-4d88-a3c4-095c1a487a56 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:12.837878 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5399281-77d8-4d88-a3c4-095c1a487a56 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:12.839389 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 68/135] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:12.840613 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5399281-77d8-4d88-a3c4-095c1a487a56 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:12.884289 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215dc6e4-7885-4688-aed5-ea22deff3052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:12.885414 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215dc6e4-7885-4688-aed5-ea22deff3052 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:12.886067 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215dc6e4-7885-4688-aed5-ea22deff3052 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:12.886350 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215dc6e4-7885-4688-aed5-ea22deff3052 None admin] PATH_INFO: `/v3/projects/e67bbf3bcff7420690a69b58a6241244` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:12.891997 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215dc6e4-7885-4688-aed5-ea22deff3052 None admin] RBAC: Authorizing `identity:get_project(project_id=e67bbf3bcff7420690a69b58a6241244)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:12.894977 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215dc6e4-7885-4688-aed5-ea22deff3052 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:12.897783 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 68/136] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:43:12 2022] GET /identity/v3/projects/e67bbf3bcff7420690a69b58a6241244 => generated 298 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:12.898973 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4425cf65-241c-486c-819e-5337643b17ba None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:12.899272 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4425cf65-241c-486c-819e-5337643b17ba None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:12.899550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4425cf65-241c-486c-819e-5337643b17ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:12.938117 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-4425cf65-241c-486c-819e-5337643b17ba None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:12.940365 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-4425cf65-241c-486c-819e-5337643b17ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:13.025950 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5399281-77d8-4d88-a3c4-095c1a487a56 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:13.064398 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a951f75-addf-4704-920c-36cf1201bc74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:13.065366 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a951f75-addf-4704-920c-36cf1201bc74 None admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:13.065683 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a951f75-addf-4704-920c-36cf1201bc74 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:13.065967 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a951f75-addf-4704-920c-36cf1201bc74 None admin] PATH_INFO: `/v3/projects/015b322c1b854846b17c9f0470487f18/users/dce468546a1841c0bc2f6bda45393ccd/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:13.068243 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a951f75-addf-4704-920c-36cf1201bc74 None admin] RBAC: Authorizing `identity:create_grant(project_id=015b322c1b854846b17c9f0470487f18, user_id=dce468546a1841c0bc2f6bda45393ccd, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:13.069999 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a951f75-addf-4704-920c-36cf1201bc74 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:13.082053 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 69/137] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:13.082937 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4425cf65-241c-486c-819e-5337643b17ba None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:13.087586 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 69/138] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jun 23 01:43:12 2022] PUT /identity/v3/projects/015b322c1b854846b17c9f0470487f18/users/dce468546a1841c0bc2f6bda45393ccd/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:43:13.125942 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a951f75-addf-4704-920c-36cf1201bc74 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:13.153939 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:13.318556 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:13.328947 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a951f75-addf-4704-920c-36cf1201bc74 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:13.383901 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4425cf65-241c-486c-819e-5337643b17ba None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:13.393418 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db6e8aa-f2cc-4307-ab65-f1e211d9eac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:13.394663 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6e8aa-f2cc-4307-ab65-f1e211d9eac1 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:13.395283 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6e8aa-f2cc-4307-ab65-f1e211d9eac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:13.395790 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6e8aa-f2cc-4307-ab65-f1e211d9eac1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:13.397427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db6e8aa-f2cc-4307-ab65-f1e211d9eac1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:13.399131 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db6e8aa-f2cc-4307-ab65-f1e211d9eac1 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:13.401152 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 70/139] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:43:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:43:13.406665 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db6e8aa-f2cc-4307-ab65-f1e211d9eac1 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:13.446274 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7bdca4-b91c-4d7d-a41d-2ccdc4363c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:13.448188 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7bdca4-b91c-4d7d-a41d-2ccdc4363c0a None admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:13.448741 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7bdca4-b91c-4d7d-a41d-2ccdc4363c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:13.449390 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7bdca4-b91c-4d7d-a41d-2ccdc4363c0a None admin] PATH_INFO: `/v3/projects/e67bbf3bcff7420690a69b58a6241244/users/dce468546a1841c0bc2f6bda45393ccd/roles/1c7adeca27524e258f7ddd2c107e43e1` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:13.454276 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7bdca4-b91c-4d7d-a41d-2ccdc4363c0a None admin] RBAC: Authorizing `identity:create_grant(project_id=e67bbf3bcff7420690a69b58a6241244, user_id=dce468546a1841c0bc2f6bda45393ccd, role_id=1c7adeca27524e258f7ddd2c107e43e1)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:13.458056 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7bdca4-b91c-4d7d-a41d-2ccdc4363c0a None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:13.484367 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 70/140] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jun 23 01:43:13 2022] PUT /identity/v3/projects/e67bbf3bcff7420690a69b58a6241244/users/dce468546a1841c0bc2f6bda45393ccd/roles/1c7adeca27524e258f7ddd2c107e43e1 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:43:13.524421 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:13.541794 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db6e8aa-f2cc-4307-ab65-f1e211d9eac1 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:13.577557 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3a253a-2bff-414c-8247-d79a9fd0199e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:13.578296 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a253a-2bff-414c-8247-d79a9fd0199e None admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:13.578584 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a253a-2bff-414c-8247-d79a9fd0199e None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:13.578922 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a253a-2bff-414c-8247-d79a9fd0199e None admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:13.579686 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3a253a-2bff-414c-8247-d79a9fd0199e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:13.581317 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3a253a-2bff-414c-8247-d79a9fd0199e None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:13.609375 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 71/141] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Jun 23 01:43:13 2022] POST /identity/v3/users => generated 283 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 01:43:14.596151 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5e215-e5e4-4d87-81e4-f79c9662c692 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:14.596644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5e215-e5e4-4d87-81e4-f79c9662c692 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:14.596644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5e215-e5e4-4d87-81e4-f79c9662c692 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:14.598258 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 71/142] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:14.604954 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebefd27c-083a-480e-b5f3-6ed4f499c637 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:14.605351 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebefd27c-083a-480e-b5f3-6ed4f499c637 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:14.605537 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebefd27c-083a-480e-b5f3-6ed4f499c637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:14.624471 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-ebefd27c-083a-480e-b5f3-6ed4f499c637 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:14.625736 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-ebefd27c-083a-480e-b5f3-6ed4f499c637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:14.708042 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 72/143] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:14.731380 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c5e215-e5e4-4d87-81e4-f79c9662c692 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:14.838882 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c5e215-e5e4-4d87-81e4-f79c9662c692 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:14.869992 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:14.875589 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fdddf2-18e6-466e-b0ca-acbd62931771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:14.876397 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdddf2-18e6-466e-b0ca-acbd62931771 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:14.876678 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdddf2-18e6-466e-b0ca-acbd62931771 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:14.876952 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdddf2-18e6-466e-b0ca-acbd62931771 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:14.888354 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fdddf2-18e6-466e-b0ca-acbd62931771 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:14.897082 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-66fdddf2-18e6-466e-b0ca-acbd62931771 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 01:43:14.897082 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:14.897082 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:14.897082 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:14.897082 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:14.897082 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:14.897082 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:14.897082 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:14.897082 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:14.897082 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:14.897082 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:14.897082 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:14.897082 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:14.897082 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:14.897082 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:14.897082 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:14.897930 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:14.897930 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:14.897930 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:14.897930 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:14.897930 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:14.897930 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:14.897930 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:14.897930 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:14.897930 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:14.897930 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:14.897930 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:14.897930 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:14.897930 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:14.897930 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:14.897930 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:14.897930 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:14.899343 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:14.899343 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:14.899343 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:14.899343 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:14.899343 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:14.899343 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:14.899343 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:14.899343 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:14.899343 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:14.899343 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:14.899343 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 01:43:14.899343 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:43:14.899343 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 72/144] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:43:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:43:14.904412 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebefd27c-083a-480e-b5f3-6ed4f499c637 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:14.973551 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:15.008992 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebefd27c-083a-480e-b5f3-6ed4f499c637 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:15.046877 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a8e8b-5e9b-4b9e-a081-e0ccda857e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:15.047629 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a8e8b-5e9b-4b9e-a081-e0ccda857e7e None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:15.047915 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a8e8b-5e9b-4b9e-a081-e0ccda857e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:15.048200 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a8e8b-5e9b-4b9e-a081-e0ccda857e7e None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:15.048958 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277a8e8b-5e9b-4b9e-a081-e0ccda857e7e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:15.049267 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277a8e8b-5e9b-4b9e-a081-e0ccda857e7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:15.050686 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277a8e8b-5e9b-4b9e-a081-e0ccda857e7e None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:15.059972 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 73/145] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jun 23 01:43:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:15.065461 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fdddf2-18e6-466e-b0ca-acbd62931771 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:15.168919 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fdddf2-18e6-466e-b0ca-acbd62931771 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:15.204077 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a9da6a-a2b5-4a75-a5b9-2f762d3f5125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:15.205004 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9da6a-a2b5-4a75-a5b9-2f762d3f5125 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:15.205288 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9da6a-a2b5-4a75-a5b9-2f762d3f5125 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:15.205557 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9da6a-a2b5-4a75-a5b9-2f762d3f5125 None admin] PATH_INFO: `/v3/users/9df85dac31484bdbaa8f0702a4d52e5a` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:15.206779 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a9da6a-a2b5-4a75-a5b9-2f762d3f5125 None admin] RBAC: Authorizing `identity:get_user(user_id=9df85dac31484bdbaa8f0702a4d52e5a)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:15.208289 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a9da6a-a2b5-4a75-a5b9-2f762d3f5125 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:15.209983 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 73/146] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:43:15 2022] GET /identity/v3/users/9df85dac31484bdbaa8f0702a4d52e5a => generated 289 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:15.215537 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a8e8b-5e9b-4b9e-a081-e0ccda857e7e None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:15.224795 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b21974-d640-47e0-a4f4-27f4ff411433 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:15.225076 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b21974-d640-47e0-a4f4-27f4ff411433 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:15.225367 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b21974-d640-47e0-a4f4-27f4ff411433 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:15.227050 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 74/147] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:15.232427 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e47ff9-d1df-448d-9784-06b73bd0f5d5 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:15.232718 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e47ff9-d1df-448d-9784-06b73bd0f5d5 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:15.232994 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e47ff9-d1df-448d-9784-06b73bd0f5d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:15.268298 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-f7e47ff9-d1df-448d-9784-06b73bd0f5d5 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:15.269525 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-f7e47ff9-d1df-448d-9784-06b73bd0f5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:15.333957 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a8e8b-5e9b-4b9e-a081-e0ccda857e7e None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:15.376193 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e8d46-e073-444b-9a3c-4e302682877c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:15.377099 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e8d46-e073-444b-9a3c-4e302682877c None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:15.377393 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e8d46-e073-444b-9a3c-4e302682877c None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:15.377678 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e8d46-e073-444b-9a3c-4e302682877c None admin] PATH_INFO: `/v3/projects/e67bbf3bcff7420690a69b58a6241244` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:15.379431 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4e8d46-e073-444b-9a3c-4e302682877c None admin] RBAC: Authorizing `identity:get_project(project_id=e67bbf3bcff7420690a69b58a6241244)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:15.381306 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4e8d46-e073-444b-9a3c-4e302682877c None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:15.383086 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 74/148] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:43:15 2022] GET /identity/v3/projects/e67bbf3bcff7420690a69b58a6241244 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:15.383660 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:15.388791 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e8d46-e073-444b-9a3c-4e302682877c None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:15.430646 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:15.451523 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 75/149] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 01:43:15.484598 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e47ff9-d1df-448d-9784-06b73bd0f5d5 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:15.502091 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e8d46-e073-444b-9a3c-4e302682877c None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:15.541619 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8097df40-d043-42b6-ad86-c6ecc0bcb8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:15.542528 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097df40-d043-42b6-ad86-c6ecc0bcb8ed None admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:15.542817 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097df40-d043-42b6-ad86-c6ecc0bcb8ed None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:15.543095 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097df40-d043-42b6-ad86-c6ecc0bcb8ed None admin] PATH_INFO: `/v3/projects/e67bbf3bcff7420690a69b58a6241244/users/9df85dac31484bdbaa8f0702a4d52e5a/roles/d159e6edeef94e418a4616504e214063` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:15.545216 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8097df40-d043-42b6-ad86-c6ecc0bcb8ed None admin] RBAC: Authorizing `identity:create_grant(project_id=e67bbf3bcff7420690a69b58a6241244, user_id=9df85dac31484bdbaa8f0702a4d52e5a, role_id=d159e6edeef94e418a4616504e214063)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:15.546898 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8097df40-d043-42b6-ad86-c6ecc0bcb8ed None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:15.579865 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 75/150] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jun 23 01:43:15 2022] PUT /identity/v3/projects/e67bbf3bcff7420690a69b58a6241244/users/9df85dac31484bdbaa8f0702a4d52e5a/roles/d159e6edeef94e418a4616504e214063 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 01:43:15.644815 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e47ff9-d1df-448d-9784-06b73bd0f5d5 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:15.682325 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb02511-870d-4f32-b547-ba1aadcfe1b2 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:15.682821 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb02511-870d-4f32-b547-ba1aadcfe1b2 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:15.683024 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb02511-870d-4f32-b547-ba1aadcfe1b2 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:15.684597 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 76/151] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 01:43:15.691353 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ff79a-b5cb-4247-ae5d-a9ab82d4011f None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:15.691630 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ff79a-b5cb-4247-ae5d-a9ab82d4011f None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:15.691913 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ff79a-b5cb-4247-ae5d-a9ab82d4011f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:15.716751 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-d50ff79a-b5cb-4247-ae5d-a9ab82d4011f None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:15.718095 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-d50ff79a-b5cb-4247-ae5d-a9ab82d4011f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:15.724681 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1206a9-761d-400a-b117-46aae57fb32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:15.726843 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1206a9-761d-400a-b117-46aae57fb32f None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:15.726843 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1206a9-761d-400a-b117-46aae57fb32f None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:15.726843 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1206a9-761d-400a-b117-46aae57fb32f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:15.748769 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1206a9-761d-400a-b117-46aae57fb32f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:15.761081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-5c1206a9-761d-400a-b117-46aae57fb32f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 01:43:15.761081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:15.761081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:15.761081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:15.761081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:15.761081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:15.761081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:15.761081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:15.761081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:15.761081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:15.761081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:15.761081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:15.761081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:15.761081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:15.761081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:15.761081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:15.763143 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:15.763143 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:15.763143 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:15.763143 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:15.763143 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:15.763143 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:15.763143 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:15.763143 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:15.763143 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:15.763143 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:15.763143 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:15.763143 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:15.763143 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:15.763143 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:15.763143 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:15.763143 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:15.764545 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:15.764545 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:15.764545 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:15.764545 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:15.764545 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:15.764545 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:15.764545 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:15.764545 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:15.764545 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:15.764545 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:15.764545 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 01:43:15.764545 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:43:15.764545 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 76/152] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:43:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:43:15.768984 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1206a9-761d-400a-b117-46aae57fb32f None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:15.826301 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:15.843343 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 77/153] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 01:43:15.889344 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:15.891095 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50ff79a-b5cb-4247-ae5d-a9ab82d4011f None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:15.997589 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1206a9-761d-400a-b117-46aae57fb32f None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:16.054168 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50ff79a-b5cb-4247-ae5d-a9ab82d4011f None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:16.068901 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b213f8c3-dc9b-4fac-b6c1-45bf7a25495f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:16.069582 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b213f8c3-dc9b-4fac-b6c1-45bf7a25495f None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:16.069865 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b213f8c3-dc9b-4fac-b6c1-45bf7a25495f None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:16.070143 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b213f8c3-dc9b-4fac-b6c1-45bf7a25495f None admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:16.070918 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b213f8c3-dc9b-4fac-b6c1-45bf7a25495f None admin] RBAC: Adding query filter params (name=member) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:16.071265 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b213f8c3-dc9b-4fac-b6c1-45bf7a25495f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:16.072695 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b213f8c3-dc9b-4fac-b6c1-45bf7a25495f None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:16.085574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 77/154] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jun 23 01:43:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:16.091016 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b213f8c3-dc9b-4fac-b6c1-45bf7a25495f None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:16.119459 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef245d8-c160-4df8-85a8-3783140a87df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:16.120309 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef245d8-c160-4df8-85a8-3783140a87df None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:16.120590 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef245d8-c160-4df8-85a8-3783140a87df None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:16.120864 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef245d8-c160-4df8-85a8-3783140a87df None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:16.143551 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef245d8-c160-4df8-85a8-3783140a87df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:16.155440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-bef245d8-c160-4df8-85a8-3783140a87df None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:43:16.155440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:16.155440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:16.155440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:16.155440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:16.155440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:16.155440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:16.155440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:16.155440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:16.155440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:16.155440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:16.155440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:16.155440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:16.155440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:16.155440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:16.155440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:16.156318 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:16.156318 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:16.156318 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:16.156318 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:16.156318 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:16.156318 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:16.156318 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:16.156318 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:16.156318 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:16.156318 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:16.156318 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:16.156318 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:16.156318 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:16.156318 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:16.156318 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:16.156318 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:16.157816 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:16.157816 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:16.157816 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:16.157816 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:16.157816 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:16.157816 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:16.157816 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:16.157816 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:16.157816 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:16.157816 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:16.157816 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:43:16.157816 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:43:16.157816 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 78/155] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jun 23 01:43:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:43:16.162341 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef245d8-c160-4df8-85a8-3783140a87df None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:16.228750 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b213f8c3-dc9b-4fac-b6c1-45bf7a25495f None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:16.249478 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:16.295553 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32494526-6df5-4995-a62b-efa2779fff8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:16.296655 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32494526-6df5-4995-a62b-efa2779fff8c None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:16.297108 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32494526-6df5-4995-a62b-efa2779fff8c None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:16.297575 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32494526-6df5-4995-a62b-efa2779fff8c None admin] PATH_INFO: `/v3/users/dce468546a1841c0bc2f6bda45393ccd` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:16.297675 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:16.303984 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32494526-6df5-4995-a62b-efa2779fff8c None admin] RBAC: Authorizing `identity:get_user(user_id=dce468546a1841c0bc2f6bda45393ccd)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:16.305995 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32494526-6df5-4995-a62b-efa2779fff8c None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:16.307907 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 78/156] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:43:16 2022] GET /identity/v3/users/dce468546a1841c0bc2f6bda45393ccd => generated 275 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:43:16.313466 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32494526-6df5-4995-a62b-efa2779fff8c None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:16.371240 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef245d8-c160-4df8-85a8-3783140a87df None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:16.416569 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b243e14-8ad5-4545-9505-897cdc6d7fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:16.417256 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b243e14-8ad5-4545-9505-897cdc6d7fdb None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:16.417536 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b243e14-8ad5-4545-9505-897cdc6d7fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:16.417808 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b243e14-8ad5-4545-9505-897cdc6d7fdb None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:16.418572 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b243e14-8ad5-4545-9505-897cdc6d7fdb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:16.418866 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b243e14-8ad5-4545-9505-897cdc6d7fdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:16.420310 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b243e14-8ad5-4545-9505-897cdc6d7fdb None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:16.433478 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 79/157] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:43:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:43:16.439121 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b243e14-8ad5-4545-9505-897cdc6d7fdb None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:16.481840 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32494526-6df5-4995-a62b-efa2779fff8c None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:16.532999 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a27d8-33f1-43b2-bf98-44937cf754d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:16.533579 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a27d8-33f1-43b2-bf98-44937cf754d5 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:16.533861 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a27d8-33f1-43b2-bf98-44937cf754d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:16.534142 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a27d8-33f1-43b2-bf98-44937cf754d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:16.584848 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b243e14-8ad5-4545-9505-897cdc6d7fdb None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:16.613067 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:16.615648 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:16.635390 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5a27d8-33f1-43b2-bf98-44937cf754d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:16.636888 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5a27d8-33f1-43b2-bf98-44937cf754d5 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:16.656493 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af2dec-96b3-4454-9593-0e0666fe98a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:16.657398 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af2dec-96b3-4454-9593-0e0666fe98a8 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:16.657670 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af2dec-96b3-4454-9593-0e0666fe98a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:16.657937 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af2dec-96b3-4454-9593-0e0666fe98a8 None admin] PATH_INFO: `/v3/users/34e5aabcf8f5425f9c9f312f635ff77a` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:16.659185 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75af2dec-96b3-4454-9593-0e0666fe98a8 None admin] RBAC: Authorizing `identity:get_user(user_id=34e5aabcf8f5425f9c9f312f635ff77a)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:16.660686 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75af2dec-96b3-4454-9593-0e0666fe98a8 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:16.662682 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 80/158] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:43:16 2022] GET /identity/v3/users/34e5aabcf8f5425f9c9f312f635ff77a => generated 247 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:43:16.672271 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af2dec-96b3-4454-9593-0e0666fe98a8 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:16.796529 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 79/159] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:43:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 01:43:16.797629 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f8714-f03f-4408-ad3f-e57d6742c7fc None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:16.803736 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f8714-f03f-4408-ad3f-e57d6742c7fc None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:16.804142 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f8714-f03f-4408-ad3f-e57d6742c7fc None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:16.805566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 80/160] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:16 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:16.806623 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801f8714-f03f-4408-ad3f-e57d6742c7fc None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:16.839600 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af2dec-96b3-4454-9593-0e0666fe98a8 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:16.878587 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac6b2df-7b6e-46c8-bfd9-b3efdaab5da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:16.879268 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6b2df-7b6e-46c8-bfd9-b3efdaab5da2 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:16.879548 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6b2df-7b6e-46c8-bfd9-b3efdaab5da2 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:16.879859 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6b2df-7b6e-46c8-bfd9-b3efdaab5da2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:16.927371 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:16.955400 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801f8714-f03f-4408-ad3f-e57d6742c7fc None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:16.985970 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:17.012894 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2aee45-379e-4c8c-b476-88e6bbc3d922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:17.014077 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2aee45-379e-4c8c-b476-88e6bbc3d922 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:17.014573 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2aee45-379e-4c8c-b476-88e6bbc3d922 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:17.015030 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2aee45-379e-4c8c-b476-88e6bbc3d922 None admin] PATH_INFO: `/v3/projects/e67bbf3bcff7420690a69b58a6241244` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:17.016482 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2aee45-379e-4c8c-b476-88e6bbc3d922 None admin] RBAC: Authorizing `identity:get_project(project_id=e67bbf3bcff7420690a69b58a6241244)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:17.018120 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2aee45-379e-4c8c-b476-88e6bbc3d922 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:17.020117 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 81/161] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:43:16 2022] GET /identity/v3/projects/e67bbf3bcff7420690a69b58a6241244 => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:17.020717 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac6b2df-7b6e-46c8-bfd9-b3efdaab5da2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:17.027250 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac6b2df-7b6e-46c8-bfd9-b3efdaab5da2 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:17.027738 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b41b7c9-c8be-4907-8659-775d6c5eb456 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:17.028205 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b41b7c9-c8be-4907-8659-775d6c5eb456 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:17.028647 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b41b7c9-c8be-4907-8659-775d6c5eb456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:17.050813 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-5b41b7c9-c8be-4907-8659-775d6c5eb456 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:17.052460 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-5b41b7c9-c8be-4907-8659-775d6c5eb456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:17.182539 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 81/162] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:43:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 01:43:17.183432 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac6b2df-7b6e-46c8-bfd9-b3efdaab5da2 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:17.198087 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 82/163] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:17.199192 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b41b7c9-c8be-4907-8659-775d6c5eb456 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:17.303642 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:17.341072 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac6b2df-7b6e-46c8-bfd9-b3efdaab5da2 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:17.380871 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd38b98-5faf-4a10-b358-76391cb8d5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:17.381538 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd38b98-5faf-4a10-b358-76391cb8d5dd None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:17.381816 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd38b98-5faf-4a10-b358-76391cb8d5dd None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:17.382090 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd38b98-5faf-4a10-b358-76391cb8d5dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:17.382933 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd38b98-5faf-4a10-b358-76391cb8d5dd None admin] RBAC: Adding query filter params (role.id=9021e3c4e3384a309951be4cc1d852e5, scope.project.id=e67bbf3bcff7420690a69b58a6241244, user.id=dce468546a1841c0bc2f6bda45393ccd) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:17.383265 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd38b98-5faf-4a10-b358-76391cb8d5dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:17.384710 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd38b98-5faf-4a10-b358-76391cb8d5dd None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:17.390803 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 82/164] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:43:17 2022] GET /identity/v3/role_assignments?user.id=dce468546a1841c0bc2f6bda45393ccd&scope.project.id=e67bbf3bcff7420690a69b58a6241244&role.id=9021e3c4e3384a309951be4cc1d852e5 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:17.399603 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:17.399911 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd38b98-5faf-4a10-b358-76391cb8d5dd None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:17.441736 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b41b7c9-c8be-4907-8659-775d6c5eb456 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:17.532188 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94af2c3c-ad9c-4079-86d3-89f4de425151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:17.533062 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af2c3c-ad9c-4079-86d3-89f4de425151 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:17.533345 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af2c3c-ad9c-4079-86d3-89f4de425151 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:17.533675 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af2c3c-ad9c-4079-86d3-89f4de425151 None admin] PATH_INFO: `/v3/projects/e67bbf3bcff7420690a69b58a6241244` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:17.534926 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94af2c3c-ad9c-4079-86d3-89f4de425151 None admin] RBAC: Authorizing `identity:get_project(project_id=e67bbf3bcff7420690a69b58a6241244)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:17.536440 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94af2c3c-ad9c-4079-86d3-89f4de425151 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:17.543514 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 83/165] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:43:17 2022] GET /identity/v3/projects/e67bbf3bcff7420690a69b58a6241244 => generated 298 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:17.547149 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94af2c3c-ad9c-4079-86d3-89f4de425151 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:17.649407 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd38b98-5faf-4a10-b358-76391cb8d5dd None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:17.693115 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94af2c3c-ad9c-4079-86d3-89f4de425151 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:17.728036 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7961fd-3c9d-4e99-8706-ea17d2aef06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:17.728874 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7961fd-3c9d-4e99-8706-ea17d2aef06e None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:17.729165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7961fd-3c9d-4e99-8706-ea17d2aef06e None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:17.729447 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7961fd-3c9d-4e99-8706-ea17d2aef06e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:17.750841 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bad329-79b5-4618-8f28-0cb17570ebac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:17.751552 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:17.751656 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bad329-79b5-4618-8f28-0cb17570ebac None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:17.751825 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bad329-79b5-4618-8f28-0cb17570ebac None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:17.752151 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bad329-79b5-4618-8f28-0cb17570ebac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:17.753011 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bad329-79b5-4618-8f28-0cb17570ebac None admin] RBAC: Adding query filter params (role.id=7b20d65f49ee4a48894b6c060bb5d580, scope.project.id=e67bbf3bcff7420690a69b58a6241244, user.id=34e5aabcf8f5425f9c9f312f635ff77a) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:17.753374 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bad329-79b5-4618-8f28-0cb17570ebac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:17.754929 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bad329-79b5-4618-8f28-0cb17570ebac None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:17.759531 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7961fd-3c9d-4e99-8706-ea17d2aef06e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:17.765293 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 84/166] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:43:17 2022] GET /identity/v3/role_assignments?user.id=34e5aabcf8f5425f9c9f312f635ff77a&scope.project.id=e67bbf3bcff7420690a69b58a6241244&role.id=7b20d65f49ee4a48894b6c060bb5d580 => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:43:17.767633 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-be7961fd-3c9d-4e99-8706-ea17d2aef06e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 01:43:17.767633 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:17.767633 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:17.767633 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:17.767633 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:17.767633 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:17.767633 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:17.767633 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:17.767633 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:17.767633 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:17.767633 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:17.767633 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:17.767633 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:17.767633 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:17.767633 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:17.767633 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:17.768528 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:17.768528 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:17.768528 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:17.768528 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:17.768528 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:17.768528 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:17.768528 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:17.768528 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:17.768528 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:17.768528 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:17.768528 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:17.768528 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:17.768528 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:17.768528 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:17.768528 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:17.768528 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:17.769856 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:17.769856 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:17.769856 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:17.769856 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:17.769856 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:17.769856 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:17.769856 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:17.769856 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:17.769856 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:17.769856 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:17.769856 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 01:43:17.769856 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:43:17.769856 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 83/167] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:43:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:43:17.777722 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bad329-79b5-4618-8f28-0cb17570ebac None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:17.817217 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:17.857425 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d8d18-75b5-4936-aaa4-e0e9cb80acd8 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:17.858003 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d8d18-75b5-4936-aaa4-e0e9cb80acd8 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:17.858003 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d8d18-75b5-4936-aaa4-e0e9cb80acd8 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:17.859566 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 84/168] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:17.865199 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a52042-fc93-4831-9c17-3cefc2f66657 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:17.865487 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a52042-fc93-4831-9c17-3cefc2f66657 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:17.865761 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a52042-fc93-4831-9c17-3cefc2f66657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:17.901500 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-65a52042-fc93-4831-9c17-3cefc2f66657 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:17.902652 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-65a52042-fc93-4831-9c17-3cefc2f66657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:17.915714 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bad329-79b5-4618-8f28-0cb17570ebac None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:17.953976 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb7857-379b-4ffc-afb3-b75a33ec951c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:17.955239 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb7857-379b-4ffc-afb3-b75a33ec951c None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:17.955536 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb7857-379b-4ffc-afb3-b75a33ec951c None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:17.955808 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb7857-379b-4ffc-afb3-b75a33ec951c None admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:17.956790 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bb7857-379b-4ffc-afb3-b75a33ec951c None admin] RBAC: Adding query filter params (name=member) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:17.957095 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bb7857-379b-4ffc-afb3-b75a33ec951c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:17.958643 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bb7857-379b-4ffc-afb3-b75a33ec951c None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:17.968775 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 85/169] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jun 23 01:43:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:17.975165 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb7857-379b-4ffc-afb3-b75a33ec951c None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:18.069014 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:18.091233 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb7857-379b-4ffc-afb3-b75a33ec951c None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:18.119166 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 85/170] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 01:43:18.130970 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9a39ee-7a0f-409f-9a70-f946f41798c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:18.131901 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a39ee-7a0f-409f-9a70-f946f41798c1 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:18.132188 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a39ee-7a0f-409f-9a70-f946f41798c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:18.132470 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a39ee-7a0f-409f-9a70-f946f41798c1 None admin] PATH_INFO: `/v3/users/dce468546a1841c0bc2f6bda45393ccd` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:18.133718 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9a39ee-7a0f-409f-9a70-f946f41798c1 None admin] RBAC: Authorizing `identity:get_user(user_id=dce468546a1841c0bc2f6bda45393ccd)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:18.135515 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9a39ee-7a0f-409f-9a70-f946f41798c1 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:18.137465 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 86/171] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:43:17 2022] GET /identity/v3/users/dce468546a1841c0bc2f6bda45393ccd => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:43:18.143060 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a52042-fc93-4831-9c17-3cefc2f66657 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:18.159796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9a39ee-7a0f-409f-9a70-f946f41798c1 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:18.308713 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a52042-fc93-4831-9c17-3cefc2f66657 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:18.341684 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:18.348635 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedaabfa-db60-47c4-bed0-0b8991099a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:18.349830 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedaabfa-db60-47c4-bed0-0b8991099a66 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:18.350478 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedaabfa-db60-47c4-bed0-0b8991099a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:18.350866 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedaabfa-db60-47c4-bed0-0b8991099a66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:18.357656 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9a39ee-7a0f-409f-9a70-f946f41798c1 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:18.418674 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9736ce-1cb8-4a18-9520-195bcf8a9334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:18.419827 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9736ce-1cb8-4a18-9520-195bcf8a9334 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:18.420336 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9736ce-1cb8-4a18-9520-195bcf8a9334 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:18.420757 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9736ce-1cb8-4a18-9520-195bcf8a9334 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:18.434200 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedaabfa-db60-47c4-bed0-0b8991099a66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:18.436545 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedaabfa-db60-47c4-bed0-0b8991099a66 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:18.446041 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:18.457875 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9736ce-1cb8-4a18-9520-195bcf8a9334 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:18.466602 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-bd9736ce-1cb8-4a18-9520-195bcf8a9334 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 01:43:18.466602 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:18.466602 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:18.466602 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:18.466602 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:18.466602 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:18.466602 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:18.466602 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:18.466602 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:18.466602 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:18.466602 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:18.466602 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:18.466602 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:18.466602 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:18.466602 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:18.466602 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:18.467683 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:18.467683 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:18.467683 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:18.467683 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:18.467683 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:18.467683 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:18.467683 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:18.467683 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:18.467683 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:18.467683 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:18.467683 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:18.467683 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:18.467683 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:18.467683 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:18.467683 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:18.467683 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:18.468532 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:18.468532 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:18.468532 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:18.468532 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:18.468532 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:18.468532 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:18.468532 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:18.468532 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:18.468532 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:18.468532 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:18.468532 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 01:43:18.468532 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:43:18.471670 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 87/172] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Jun 23 01:43:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:43:18.480184 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9736ce-1cb8-4a18-9520-195bcf8a9334 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:18.557295 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 86/173] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:43:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 01:43:18.563497 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedaabfa-db60-47c4-bed0-0b8991099a66 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:18.600257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:18.637531 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9736ce-1cb8-4a18-9520-195bcf8a9334 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:18.680488 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d2898-d6fd-4255-854f-3aa228b49a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:18.681806 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d2898-d6fd-4255-854f-3aa228b49a3f None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:18.682239 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d2898-d6fd-4255-854f-3aa228b49a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:18.682601 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedaabfa-db60-47c4-bed0-0b8991099a66 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:18.682805 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d2898-d6fd-4255-854f-3aa228b49a3f None admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:18.683731 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891d2898-d6fd-4255-854f-3aa228b49a3f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:18.684264 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891d2898-d6fd-4255-854f-3aa228b49a3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:18.686012 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891d2898-d6fd-4255-854f-3aa228b49a3f None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:18.695242 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 88/174] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:43:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:18.696783 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb610f0-bf7a-472b-97ab-b7c783bddd6b None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:18.701583 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb610f0-bf7a-472b-97ab-b7c783bddd6b None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:18.702041 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb610f0-bf7a-472b-97ab-b7c783bddd6b None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:18.703772 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 89/175] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:18.705108 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b4d22-956d-412e-9391-c3dae69649e5 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:18.705532 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b4d22-956d-412e-9391-c3dae69649e5 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:18.705946 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b4d22-956d-412e-9391-c3dae69649e5 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:18.707554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 90/176] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:18.712341 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611b4d22-956d-412e-9391-c3dae69649e5 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:18.723488 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc63dda3-28b1-4b21-b99f-25ef83720b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:18.724426 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc63dda3-28b1-4b21-b99f-25ef83720b3f None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:18.724701 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc63dda3-28b1-4b21-b99f-25ef83720b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:18.724974 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc63dda3-28b1-4b21-b99f-25ef83720b3f None admin] PATH_INFO: `/v3/projects/015b322c1b854846b17c9f0470487f18` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:18.726270 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc63dda3-28b1-4b21-b99f-25ef83720b3f None admin] RBAC: Authorizing `identity:get_project(project_id=015b322c1b854846b17c9f0470487f18)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:18.727856 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc63dda3-28b1-4b21-b99f-25ef83720b3f None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:18.729561 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 87/177] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:43:18 2022] GET /identity/v3/projects/015b322c1b854846b17c9f0470487f18 => generated 312 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:18.730986 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877aaa6-682e-40dd-aada-8b9e0a83c5cc None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:18.731279 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877aaa6-682e-40dd-aada-8b9e0a83c5cc None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:18.731553 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877aaa6-682e-40dd-aada-8b9e0a83c5cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:18.749188 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-8877aaa6-682e-40dd-aada-8b9e0a83c5cc None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:18.750569 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-8877aaa6-682e-40dd-aada-8b9e0a83c5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:18.781479 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:18.831234 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 88/178] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:18.831234 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df4d9d-30dd-4f8e-8441-f19314dde084 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:18.831234 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df4d9d-30dd-4f8e-8441-f19314dde084 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:18.831234 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df4d9d-30dd-4f8e-8441-f19314dde084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:18.831849 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611b4d22-956d-412e-9391-c3dae69649e5 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:18.857218 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-f5df4d9d-30dd-4f8e-8441-f19314dde084 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:18.858887 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-f5df4d9d-30dd-4f8e-8441-f19314dde084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:18.886158 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:18.890917 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff8434-39aa-42cc-9f6b-cbba3e67bbcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:18.891823 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff8434-39aa-42cc-9f6b-cbba3e67bbcb None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:18.892118 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff8434-39aa-42cc-9f6b-cbba3e67bbcb None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:18.892516 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff8434-39aa-42cc-9f6b-cbba3e67bbcb None admin] PATH_INFO: `/v3/users/dce468546a1841c0bc2f6bda45393ccd` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:18.894062 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ff8434-39aa-42cc-9f6b-cbba3e67bbcb None admin] RBAC: Authorizing `identity:get_user(user_id=dce468546a1841c0bc2f6bda45393ccd)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:18.895808 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ff8434-39aa-42cc-9f6b-cbba3e67bbcb None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:18.902950 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 91/179] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:43:18 2022] GET /identity/v3/users/dce468546a1841c0bc2f6bda45393ccd => generated 275 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:18.902950 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff8434-39aa-42cc-9f6b-cbba3e67bbcb None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:18.943125 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 89/180] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:18.946016 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df4d9d-30dd-4f8e-8441-f19314dde084 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:19.044000 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff8434-39aa-42cc-9f6b-cbba3e67bbcb None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:19.074437 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df4d9d-30dd-4f8e-8441-f19314dde084 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:19.076703 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:19.084445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1474222-2942-45f0-98b4-9a37de2632bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:19.085502 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1474222-2942-45f0-98b4-9a37de2632bc None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:19.085980 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1474222-2942-45f0-98b4-9a37de2632bc None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:19.086447 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1474222-2942-45f0-98b4-9a37de2632bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:19.087612 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1474222-2942-45f0-98b4-9a37de2632bc None admin] RBAC: Adding query filter params (role.id=9021e3c4e3384a309951be4cc1d852e5, scope.project.id=015b322c1b854846b17c9f0470487f18, user.id=dce468546a1841c0bc2f6bda45393ccd) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:19.088122 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1474222-2942-45f0-98b4-9a37de2632bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:19.089916 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1474222-2942-45f0-98b4-9a37de2632bc None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:19.096427 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 92/181] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:43:18 2022] GET /identity/v3/role_assignments?user.id=dce468546a1841c0bc2f6bda45393ccd&scope.project.id=015b322c1b854846b17c9f0470487f18&role.id=9021e3c4e3384a309951be4cc1d852e5 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:19.097594 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1474222-2942-45f0-98b4-9a37de2632bc None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:19.119143 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd7d113-aa6a-4921-987e-a4ec9f0b225a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:19.120443 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd7d113-aa6a-4921-987e-a4ec9f0b225a None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:19.120985 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd7d113-aa6a-4921-987e-a4ec9f0b225a None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:19.121452 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd7d113-aa6a-4921-987e-a4ec9f0b225a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:19.153269 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd7d113-aa6a-4921-987e-a4ec9f0b225a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:19.168513 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-abd7d113-aa6a-4921-987e-a4ec9f0b225a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:43:19.168513 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:19.168513 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:19.168513 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:19.168513 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:19.168513 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:19.168513 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:19.168513 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:19.168513 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:19.168513 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:19.168513 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:19.168513 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:19.168513 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:19.168513 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:19.168513 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:19.168513 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:19.169489 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:19.169489 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:19.169489 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:19.169489 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:19.169489 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:19.169489 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:19.169489 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:19.169489 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:19.169489 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:19.169489 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:19.169489 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:19.169489 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:19.169489 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:19.169489 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:19.169489 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:19.169489 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:19.170258 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:19.170258 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:19.170258 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:19.170258 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:19.170258 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:19.170258 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:19.170258 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:19.170258 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:19.170258 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:19.170258 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:19.170258 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:43:19.170258 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:43:19.183485 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 90/182] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jun 23 01:43:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:43:19.184856 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd7d113-aa6a-4921-987e-a4ec9f0b225a None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:19.219977 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:19.292660 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1474222-2942-45f0-98b4-9a37de2632bc None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:19.334873 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19af0d5-c5fe-40b6-bde7-7d40cc230aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:19.335566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19af0d5-c5fe-40b6-bde7-7d40cc230aaa None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:19.335849 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19af0d5-c5fe-40b6-bde7-7d40cc230aaa None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:19.336124 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19af0d5-c5fe-40b6-bde7-7d40cc230aaa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:19.343254 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd7d113-aa6a-4921-987e-a4ec9f0b225a None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:19.381474 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45662a9-39d4-4181-a034-6f2e903fbb7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:19.382296 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45662a9-39d4-4181-a034-6f2e903fbb7d None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:19.382575 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45662a9-39d4-4181-a034-6f2e903fbb7d None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:19.382850 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45662a9-39d4-4181-a034-6f2e903fbb7d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:19.393115 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45662a9-39d4-4181-a034-6f2e903fbb7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:19.397529 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:19.401394 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-a45662a9-39d4-4181-a034-6f2e903fbb7d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 01:43:19.401394 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:19.401394 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:19.401394 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:19.401394 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:19.401394 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:19.401394 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:19.401394 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:19.401394 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:19.401394 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:19.401394 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:19.401394 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:19.401394 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:19.401394 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:19.401394 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:19.401394 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:19.402239 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:19.402239 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:19.402239 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:19.402239 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:19.402239 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:19.402239 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:19.402239 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:19.402239 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:19.402239 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:19.402239 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:19.402239 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:19.402239 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:19.402239 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:19.402239 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:19.402239 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:19.402239 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:19.403026 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:19.403026 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:19.403026 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:19.403026 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:19.403026 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:19.403026 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:19.403026 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:19.403026 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:19.403026 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:19.403026 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:19.403026 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 01:43:19.403026 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:43:19.403026 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 91/183] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Jun 23 01:43:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:43:19.403708 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45662a9-39d4-4181-a034-6f2e903fbb7d None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:19.407644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19af0d5-c5fe-40b6-bde7-7d40cc230aaa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:19.410265 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19af0d5-c5fe-40b6-bde7-7d40cc230aaa None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:19.482503 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:19.504708 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45662a9-39d4-4181-a034-6f2e903fbb7d None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:19.519126 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 93/184] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:43:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 01:43:19.519923 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd435c-b011-44b5-b97c-1f1c299beaa4 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:19.520211 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd435c-b011-44b5-b97c-1f1c299beaa4 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:19.520469 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd435c-b011-44b5-b97c-1f1c299beaa4 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:19.522291 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 94/185] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:19.524439 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf7195-ef2b-4b03-aa4d-ae47c4890e4b None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:19.524725 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf7195-ef2b-4b03-aa4d-ae47c4890e4b None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:19.524996 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf7195-ef2b-4b03-aa4d-ae47c4890e4b None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:19.526276 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 95/186] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:19.526978 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abf7195-ef2b-4b03-aa4d-ae47c4890e4b None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:19.548368 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cdc6c1-1a7d-41ca-8178-53cff63e386e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:19.549339 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cdc6c1-1a7d-41ca-8178-53cff63e386e None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:19.549618 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cdc6c1-1a7d-41ca-8178-53cff63e386e None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:19.549873 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cdc6c1-1a7d-41ca-8178-53cff63e386e None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:19.550627 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cdc6c1-1a7d-41ca-8178-53cff63e386e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:19.550926 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cdc6c1-1a7d-41ca-8178-53cff63e386e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:19.552895 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cdc6c1-1a7d-41ca-8178-53cff63e386e None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:19.561606 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 92/187] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:43:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:19.562390 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cdc6c1-1a7d-41ca-8178-53cff63e386e None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:19.626149 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:19.634644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abf7195-ef2b-4b03-aa4d-ae47c4890e4b None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:19.660206 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cdc6c1-1a7d-41ca-8178-53cff63e386e None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:19.667545 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84819cf7-5640-4b47-b04b-9adc5b2b3992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:19.668200 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84819cf7-5640-4b47-b04b-9adc5b2b3992 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:19.668486 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84819cf7-5640-4b47-b04b-9adc5b2b3992 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:19.668729 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84819cf7-5640-4b47-b04b-9adc5b2b3992 None admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:19.669502 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84819cf7-5640-4b47-b04b-9adc5b2b3992 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:19.669802 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84819cf7-5640-4b47-b04b-9adc5b2b3992 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:19.671162 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84819cf7-5640-4b47-b04b-9adc5b2b3992 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:19.679620 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 96/188] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:43:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:19.681123 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86900ec0-0e99-4d17-8838-5318e7ab3163 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:19.681505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86900ec0-0e99-4d17-8838-5318e7ab3163 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:19.681745 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86900ec0-0e99-4d17-8838-5318e7ab3163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:19.694762 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92812459-409c-475c-aea1-c626c3731af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:19.695659 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92812459-409c-475c-aea1-c626c3731af9 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:19.695942 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92812459-409c-475c-aea1-c626c3731af9 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:19.696211 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92812459-409c-475c-aea1-c626c3731af9 None admin] PATH_INFO: `/v3/projects/e67bbf3bcff7420690a69b58a6241244` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:19.697400 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92812459-409c-475c-aea1-c626c3731af9 None admin] RBAC: Authorizing `identity:get_project(project_id=e67bbf3bcff7420690a69b58a6241244)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:19.698509 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-86900ec0-0e99-4d17-8838-5318e7ab3163 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:19.698839 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92812459-409c-475c-aea1-c626c3731af9 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:19.699719 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-86900ec0-0e99-4d17-8838-5318e7ab3163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:19.700580 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 93/189] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:43:19 2022] GET /identity/v3/projects/e67bbf3bcff7420690a69b58a6241244 => generated 298 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:19.701709 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b12fc-505a-46b6-9bc5-3dec3c670ef6 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:19.701990 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b12fc-505a-46b6-9bc5-3dec3c670ef6 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:19.702258 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b12fc-505a-46b6-9bc5-3dec3c670ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:19.717944 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-208b12fc-505a-46b6-9bc5-3dec3c670ef6 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:19.719115 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-208b12fc-505a-46b6-9bc5-3dec3c670ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:19.721056 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:19.780089 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 97/190] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:19.780089 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86900ec0-0e99-4d17-8838-5318e7ab3163 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:19.799773 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 94/191] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:19.800600 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b12fc-505a-46b6-9bc5-3dec3c670ef6 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:19.872241 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:19.886822 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86900ec0-0e99-4d17-8838-5318e7ab3163 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:19.912861 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b12fc-505a-46b6-9bc5-3dec3c670ef6 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:19.922668 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa5ddc-70e9-47d1-bcfe-cb9ec4a6b407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:19.923666 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa5ddc-70e9-47d1-bcfe-cb9ec4a6b407 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:19.923949 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa5ddc-70e9-47d1-bcfe-cb9ec4a6b407 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:19.924250 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa5ddc-70e9-47d1-bcfe-cb9ec4a6b407 None admin] PATH_INFO: `/v3/users/f8870aeb21ef413b80b1c98c50efba39` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:19.938080 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fa5ddc-70e9-47d1-bcfe-cb9ec4a6b407 None admin] RBAC: Authorizing `identity:get_user(user_id=f8870aeb21ef413b80b1c98c50efba39)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:19.939599 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fa5ddc-70e9-47d1-bcfe-cb9ec4a6b407 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:19.941274 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 98/192] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:43:19 2022] GET /identity/v3/users/f8870aeb21ef413b80b1c98c50efba39 => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:19.942039 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa5ddc-70e9-47d1-bcfe-cb9ec4a6b407 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:19.950825 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4041d-f69c-4d14-9086-073c53364f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:19.951751 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4041d-f69c-4d14-9086-073c53364f64 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:19.952042 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4041d-f69c-4d14-9086-073c53364f64 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:19.952292 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4041d-f69c-4d14-9086-073c53364f64 None admin] PATH_INFO: `/v3/users/f8870aeb21ef413b80b1c98c50efba39` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:19.953515 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a4041d-f69c-4d14-9086-073c53364f64 None admin] RBAC: Authorizing `identity:get_user(user_id=f8870aeb21ef413b80b1c98c50efba39)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:19.954947 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a4041d-f69c-4d14-9086-073c53364f64 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:19.956757 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 95/193] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:43:19 2022] GET /identity/v3/users/f8870aeb21ef413b80b1c98c50efba39 => generated 283 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:19.957446 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4041d-f69c-4d14-9086-073c53364f64 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:19.965924 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:20.041912 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa5ddc-70e9-47d1-bcfe-cb9ec4a6b407 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:20.057304 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4041d-f69c-4d14-9086-073c53364f64 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:20.074917 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7b3e20-0aa0-4503-a8fd-c027420a75de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:20.075586 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b3e20-0aa0-4503-a8fd-c027420a75de None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:20.075871 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b3e20-0aa0-4503-a8fd-c027420a75de None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:20.076145 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b3e20-0aa0-4503-a8fd-c027420a75de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:20.076990 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7b3e20-0aa0-4503-a8fd-c027420a75de None admin] RBAC: Adding query filter params (role.id=1c7adeca27524e258f7ddd2c107e43e1, scope.project.id=e67bbf3bcff7420690a69b58a6241244, user.id=dce468546a1841c0bc2f6bda45393ccd) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:20.077300 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7b3e20-0aa0-4503-a8fd-c027420a75de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:20.078767 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7b3e20-0aa0-4503-a8fd-c027420a75de None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:20.084771 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 99/194] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:43:19 2022] GET /identity/v3/role_assignments?user.id=dce468546a1841c0bc2f6bda45393ccd&scope.project.id=e67bbf3bcff7420690a69b58a6241244&role.id=1c7adeca27524e258f7ddd2c107e43e1 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:20.085410 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7b3e20-0aa0-4503-a8fd-c027420a75de None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:20.096148 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:20.106121 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d9ff73-d456-4397-87ed-e38c1271acc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:20.106910 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d9ff73-d456-4397-87ed-e38c1271acc8 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:20.107184 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d9ff73-d456-4397-87ed-e38c1271acc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:20.107484 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d9ff73-d456-4397-87ed-e38c1271acc8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:20.108755 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d9ff73-d456-4397-87ed-e38c1271acc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:20.110274 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d9ff73-d456-4397-87ed-e38c1271acc8 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:20.112025 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 96/195] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:43:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:20.112025 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d9ff73-d456-4397-87ed-e38c1271acc8 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:20.200915 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:20.211572 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7b3e20-0aa0-4503-a8fd-c027420a75de None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:20.230360 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d9ff73-d456-4397-87ed-e38c1271acc8 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:20.256358 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9549a5a0-9b14-457b-bf1f-fc9409264310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:20.257156 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549a5a0-9b14-457b-bf1f-fc9409264310 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:20.257432 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549a5a0-9b14-457b-bf1f-fc9409264310 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:20.257802 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549a5a0-9b14-457b-bf1f-fc9409264310 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:20.269984 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9549a5a0-9b14-457b-bf1f-fc9409264310 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:20.270150 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a4850c-db64-4075-8add-c83d4e53455b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:20.270771 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a4850c-db64-4075-8add-c83d4e53455b None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:20.271092 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a4850c-db64-4075-8add-c83d4e53455b None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:20.271385 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a4850c-db64-4075-8add-c83d4e53455b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:20.277831 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-9549a5a0-9b14-457b-bf1f-fc9409264310 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 01:43:20.277831 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:20.277831 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:20.277831 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:20.277831 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:20.277831 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:20.277831 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:20.277831 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:20.277831 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:20.277831 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:20.277831 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:20.277831 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:20.277831 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:20.277831 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:20.277831 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:20.277831 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:20.278940 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:20.278940 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:20.278940 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:20.278940 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:20.278940 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:20.278940 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:20.278940 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:20.278940 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:20.278940 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:20.278940 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:20.278940 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:20.278940 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:20.278940 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:20.278940 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:20.278940 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:20.278940 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:20.279949 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:20.279949 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:20.279949 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:20.279949 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:20.279949 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:20.279949 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:20.279949 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:20.279949 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:20.279949 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:20.279949 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:20.279949 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 01:43:20.279949 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:43:20.279949 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 100/196] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:43:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:43:20.281044 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9549a5a0-9b14-457b-bf1f-fc9409264310 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:20.345541 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a4850c-db64-4075-8add-c83d4e53455b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:20.347005 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a4850c-db64-4075-8add-c83d4e53455b None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:20.352108 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:20.386282 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9549a5a0-9b14-457b-bf1f-fc9409264310 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:20.418417 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:20.420328 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b4b39e-5ade-4226-b3bc-5ee1e88c77b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:20.421351 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b4b39e-5ade-4226-b3bc-5ee1e88c77b7 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:20.421644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b4b39e-5ade-4226-b3bc-5ee1e88c77b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:20.421917 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b4b39e-5ade-4226-b3bc-5ee1e88c77b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:20.444398 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 97/197] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:43:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 01:43:20.445112 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a4850c-db64-4075-8add-c83d4e53455b None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:20.488231 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b4b39e-5ade-4226-b3bc-5ee1e88c77b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:20.489730 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b4b39e-5ade-4226-b3bc-5ee1e88c77b7 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:20.544992 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a4850c-db64-4075-8add-c83d4e53455b None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:20.552542 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:20.578953 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f4b9e-c12d-4bfa-884b-2d12bb576701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:20.579645 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f4b9e-c12d-4bfa-884b-2d12bb576701 None admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:20.579930 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f4b9e-c12d-4bfa-884b-2d12bb576701 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:20.580204 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f4b9e-c12d-4bfa-884b-2d12bb576701 None admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:20.580934 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62f4b9e-c12d-4bfa-884b-2d12bb576701 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:20.582401 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62f4b9e-c12d-4bfa-884b-2d12bb576701 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:20.587892 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 101/198] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:43:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 01:43:20.588701 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b4b39e-5ade-4226-b3bc-5ee1e88c77b7 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:20.606129 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 98/199] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Jun 23 01:43:20 2022] POST /identity/v3/users => generated 297 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:43:20.606812 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f4b9e-c12d-4bfa-884b-2d12bb576701 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:20.641029 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:20.702678 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b4b39e-5ade-4226-b3bc-5ee1e88c77b7 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:20.728670 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f4b9e-c12d-4bfa-884b-2d12bb576701 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:20.741304 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02616f12-c345-4bf8-a3cf-857665fb023a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:20.742287 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02616f12-c345-4bf8-a3cf-857665fb023a None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:20.742568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02616f12-c345-4bf8-a3cf-857665fb023a None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:20.742842 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02616f12-c345-4bf8-a3cf-857665fb023a None admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:20.743828 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02616f12-c345-4bf8-a3cf-857665fb023a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:20.744549 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02616f12-c345-4bf8-a3cf-857665fb023a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:20.746011 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02616f12-c345-4bf8-a3cf-857665fb023a None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:20.759252 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 102/200] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jun 23 01:43:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:20.759252 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02616f12-c345-4bf8-a3cf-857665fb023a None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:20.773721 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e75c1-b770-438c-8bd4-3c4ba9c9f6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:20.774953 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e75c1-b770-438c-8bd4-3c4ba9c9f6e7 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:20.775283 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e75c1-b770-438c-8bd4-3c4ba9c9f6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:20.775539 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e75c1-b770-438c-8bd4-3c4ba9c9f6e7 None admin] PATH_INFO: `/v3/projects/9db43d2e143f49efbddf812fd9d03c69` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:20.778543 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2e75c1-b770-438c-8bd4-3c4ba9c9f6e7 None admin] RBAC: Authorizing `identity:get_project(project_id=9db43d2e143f49efbddf812fd9d03c69)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:20.780058 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2e75c1-b770-438c-8bd4-3c4ba9c9f6e7 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:20.782061 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 99/201] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:43:20 2022] GET /identity/v3/projects/9db43d2e143f49efbddf812fd9d03c69 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:20.782668 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e75c1-b770-438c-8bd4-3c4ba9c9f6e7 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:20.793564 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:20.882024 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02616f12-c345-4bf8-a3cf-857665fb023a None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:20.902656 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:20.906469 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e75c1-b770-438c-8bd4-3c4ba9c9f6e7 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:20.925191 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b660841c-87f7-457d-afac-18bf7ab3e97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:20.926063 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b660841c-87f7-457d-afac-18bf7ab3e97f None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:20.926338 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b660841c-87f7-457d-afac-18bf7ab3e97f None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:20.926612 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b660841c-87f7-457d-afac-18bf7ab3e97f None admin] PATH_INFO: `/v3/projects/9db43d2e143f49efbddf812fd9d03c69` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:20.928107 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b660841c-87f7-457d-afac-18bf7ab3e97f None admin] RBAC: Authorizing `identity:get_project(project_id=9db43d2e143f49efbddf812fd9d03c69)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:20.929693 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b660841c-87f7-457d-afac-18bf7ab3e97f None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:20.932002 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 103/202] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:43:20 2022] GET /identity/v3/projects/9db43d2e143f49efbddf812fd9d03c69 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:20.932769 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b660841c-87f7-457d-afac-18bf7ab3e97f None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:20.957202 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d244dde8-6da1-47e5-ba52-4bffd5089c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:20.958105 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244dde8-6da1-47e5-ba52-4bffd5089c68 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:20.958452 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244dde8-6da1-47e5-ba52-4bffd5089c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:20.959939 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244dde8-6da1-47e5-ba52-4bffd5089c68 None admin] PATH_INFO: `/v3/users/9df85dac31484bdbaa8f0702a4d52e5a` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:20.962893 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d244dde8-6da1-47e5-ba52-4bffd5089c68 None admin] RBAC: Authorizing `identity:get_user(user_id=9df85dac31484bdbaa8f0702a4d52e5a)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:20.965218 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d244dde8-6da1-47e5-ba52-4bffd5089c68 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:20.968243 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 100/203] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:43:20 2022] GET /identity/v3/users/9df85dac31484bdbaa8f0702a4d52e5a => generated 289 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:20.969692 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d244dde8-6da1-47e5-ba52-4bffd5089c68 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:21.049309 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:21.054680 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b660841c-87f7-457d-afac-18bf7ab3e97f None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:21.088355 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d244dde8-6da1-47e5-ba52-4bffd5089c68 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:21.099406 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d4d174-e518-404d-a65e-a608f894620c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:21.100525 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d4d174-e518-404d-a65e-a608f894620c None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:21.100878 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d4d174-e518-404d-a65e-a608f894620c None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:21.101217 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d4d174-e518-404d-a65e-a608f894620c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:21.102320 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d4d174-e518-404d-a65e-a608f894620c None admin] RBAC: Adding query filter params (role.id=7b20d65f49ee4a48894b6c060bb5d580, scope.project.id=9db43d2e143f49efbddf812fd9d03c69, user.id=f8870aeb21ef413b80b1c98c50efba39) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:21.102626 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d4d174-e518-404d-a65e-a608f894620c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:21.104869 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d4d174-e518-404d-a65e-a608f894620c None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:21.112575 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 104/204] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:43:20 2022] GET /identity/v3/role_assignments?user.id=f8870aeb21ef413b80b1c98c50efba39&scope.project.id=9db43d2e143f49efbddf812fd9d03c69&role.id=7b20d65f49ee4a48894b6c060bb5d580 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:21.113448 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d4d174-e518-404d-a65e-a608f894620c None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:21.140086 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2bf00a-6222-4d0a-b72d-a0d390a4f64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:21.140852 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2bf00a-6222-4d0a-b72d-a0d390a4f64c None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:21.141221 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2bf00a-6222-4d0a-b72d-a0d390a4f64c None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:21.141586 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2bf00a-6222-4d0a-b72d-a0d390a4f64c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:21.142628 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2bf00a-6222-4d0a-b72d-a0d390a4f64c None admin] RBAC: Adding query filter params (role.id=1c7adeca27524e258f7ddd2c107e43e1, scope.project.id=9db43d2e143f49efbddf812fd9d03c69, user.id=f8870aeb21ef413b80b1c98c50efba39) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:21.143036 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2bf00a-6222-4d0a-b72d-a0d390a4f64c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:21.144710 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2bf00a-6222-4d0a-b72d-a0d390a4f64c None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:21.163609 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 101/205] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:43:20 2022] GET /identity/v3/role_assignments?user.id=f8870aeb21ef413b80b1c98c50efba39&scope.project.id=9db43d2e143f49efbddf812fd9d03c69&role.id=1c7adeca27524e258f7ddd2c107e43e1 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:21.242384 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d4d174-e518-404d-a65e-a608f894620c None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:21.288051 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec93cf8-a397-4483-9027-b381f5bfb845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:21.289090 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec93cf8-a397-4483-9027-b381f5bfb845 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:21.289546 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec93cf8-a397-4483-9027-b381f5bfb845 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:21.289993 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec93cf8-a397-4483-9027-b381f5bfb845 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:21.333209 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:21.391424 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec93cf8-a397-4483-9027-b381f5bfb845 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:21.392847 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec93cf8-a397-4483-9027-b381f5bfb845 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:21.512637 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 105/206] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:43:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 01:43:21.518228 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2bf00a-6222-4d0a-b72d-a0d390a4f64c None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:21.559070 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:21.651692 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2bf00a-6222-4d0a-b72d-a0d390a4f64c None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:21.700813 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcab196-0752-4163-8957-3e78a00460e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:21.701688 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcab196-0752-4163-8957-3e78a00460e9 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:21.701973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcab196-0752-4163-8957-3e78a00460e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:21.702245 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcab196-0752-4163-8957-3e78a00460e9 None admin] PATH_INFO: `/v3/projects/e67bbf3bcff7420690a69b58a6241244` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:21.703616 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcab196-0752-4163-8957-3e78a00460e9 None admin] RBAC: Authorizing `identity:get_project(project_id=e67bbf3bcff7420690a69b58a6241244)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:21.705069 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcab196-0752-4163-8957-3e78a00460e9 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:21.707136 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 102/207] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:43:21 2022] GET /identity/v3/projects/e67bbf3bcff7420690a69b58a6241244 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:21.712601 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec93cf8-a397-4483-9027-b381f5bfb845 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:21.797360 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:21.849001 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec93cf8-a397-4483-9027-b381f5bfb845 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:21.904900 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b707bf32-a686-4c0e-9e25-e49cc309007b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:21.905517 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707bf32-a686-4c0e-9e25-e49cc309007b None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:21.905726 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707bf32-a686-4c0e-9e25-e49cc309007b None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:21.906004 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707bf32-a686-4c0e-9e25-e49cc309007b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:21.906842 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b707bf32-a686-4c0e-9e25-e49cc309007b None admin] RBAC: Adding query filter params (role.id=d159e6edeef94e418a4616504e214063, scope.project.id=e67bbf3bcff7420690a69b58a6241244, user.id=9df85dac31484bdbaa8f0702a4d52e5a) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:21.907153 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b707bf32-a686-4c0e-9e25-e49cc309007b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:21.908631 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b707bf32-a686-4c0e-9e25-e49cc309007b None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:21.916187 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 106/208] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:43:21 2022] GET /identity/v3/role_assignments?user.id=9df85dac31484bdbaa8f0702a4d52e5a&scope.project.id=e67bbf3bcff7420690a69b58a6241244&role.id=d159e6edeef94e418a4616504e214063 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:22.904044 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e5b7e-35e0-4650-bd8f-15abe1f266bc None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:22.904566 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e5b7e-35e0-4650-bd8f-15abe1f266bc None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:22.904566 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e5b7e-35e0-4650-bd8f-15abe1f266bc None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:22.905964 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 103/209] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:22.911237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379e4cf-b5d2-408f-a5b9-e76609b08821 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:22.912071 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379e4cf-b5d2-408f-a5b9-e76609b08821 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:22.912548 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379e4cf-b5d2-408f-a5b9-e76609b08821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:22.929725 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-4379e4cf-b5d2-408f-a5b9-e76609b08821 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:22.931550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-4379e4cf-b5d2-408f-a5b9-e76609b08821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:22.999705 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 107/210] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:23.023153 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e5b7e-35e0-4650-bd8f-15abe1f266bc None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:23.123316 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:23.128015 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e5b7e-35e0-4650-bd8f-15abe1f266bc None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:23.163682 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a4db1-a665-4860-acf4-5aeda0f11a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:23.164413 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a4db1-a665-4860-acf4-5aeda0f11a86 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:23.164695 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a4db1-a665-4860-acf4-5aeda0f11a86 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:23.164973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a4db1-a665-4860-acf4-5aeda0f11a86 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:23.175696 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a4db1-a665-4860-acf4-5aeda0f11a86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:23.184061 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-188a4db1-a665-4860-acf4-5aeda0f11a86 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 01:43:23.184061 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:23.184061 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:23.184061 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:23.184061 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:23.184061 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:23.184061 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:23.184061 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:23.184061 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:23.184061 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:23.184061 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:23.184061 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:23.184061 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:23.184061 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:23.184061 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:23.184061 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:23.184906 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:23.184906 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:23.184906 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:23.184906 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:23.184906 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:23.184906 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:23.184906 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:23.184906 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:23.184906 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:23.184906 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:23.184906 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:23.184906 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:23.184906 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:23.184906 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:23.184906 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:23.184906 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:23.185668 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:23.185668 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:23.185668 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:23.185668 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:23.185668 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:23.185668 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:23.185668 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:23.185668 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:23.185668 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:23.185668 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:23.185668 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 01:43:23.185668 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:43:23.185668 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 104/211] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:43:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:43:23.191049 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379e4cf-b5d2-408f-a5b9-e76609b08821 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:23.198847 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4cc315-9149-4f3e-859a-14ec911e5280 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:23.199127 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4cc315-9149-4f3e-859a-14ec911e5280 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:23.199434 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4cc315-9149-4f3e-859a-14ec911e5280 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:23.200911 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 105/212] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 01:43:23.206311 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc12dd-c769-4162-893e-19fcea9e2f9f None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:23.206680 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc12dd-c769-4162-893e-19fcea9e2f9f None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:23.206964 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc12dd-c769-4162-893e-19fcea9e2f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:23.224867 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-eabc12dd-c769-4162-893e-19fcea9e2f9f None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:23.226054 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-eabc12dd-c769-4162-893e-19fcea9e2f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:23.254625 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:23.306845 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379e4cf-b5d2-408f-a5b9-e76609b08821 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:23.309398 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 106/213] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 01:43:23.314225 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986caab-7f2f-4503-a655-e32eb6f907f9 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:23.314719 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986caab-7f2f-4503-a655-e32eb6f907f9 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:23.315163 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986caab-7f2f-4503-a655-e32eb6f907f9 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:23.316712 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 107/214] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 01:43:23.321878 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa654c1e-0c44-4f61-bccc-bb73ffc0eb78 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:23.322362 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa654c1e-0c44-4f61-bccc-bb73ffc0eb78 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:23.322825 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa654c1e-0c44-4f61-bccc-bb73ffc0eb78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:23.339831 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-fa654c1e-0c44-4f61-bccc-bb73ffc0eb78 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:23.341495 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-fa654c1e-0c44-4f61-bccc-bb73ffc0eb78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:23.343624 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea736488-310e-43be-bf33-29eea161de4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:23.344295 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea736488-310e-43be-bf33-29eea161de4d None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:23.344573 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea736488-310e-43be-bf33-29eea161de4d None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:23.344844 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea736488-310e-43be-bf33-29eea161de4d None admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:23.345600 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea736488-310e-43be-bf33-29eea161de4d None admin] RBAC: Adding query filter params (name=member) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:23.345901 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea736488-310e-43be-bf33-29eea161de4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:23.347322 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea736488-310e-43be-bf33-29eea161de4d None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:23.356170 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 108/215] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jun 23 01:43:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:23.360132 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea736488-310e-43be-bf33-29eea161de4d None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:23.416288 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 108/216] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 01:43:23.419381 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa654c1e-0c44-4f61-bccc-bb73ffc0eb78 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:23.438292 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:23.469972 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea736488-310e-43be-bf33-29eea161de4d None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:23.487271 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:23.505095 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501390be-f0af-4c1c-a364-0213714e8ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:23.505919 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501390be-f0af-4c1c-a364-0213714e8ede None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:23.506197 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501390be-f0af-4c1c-a364-0213714e8ede None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:23.506470 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501390be-f0af-4c1c-a364-0213714e8ede None admin] PATH_INFO: `/v3/domains/default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:23.507689 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501390be-f0af-4c1c-a364-0213714e8ede None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:23.509152 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501390be-f0af-4c1c-a364-0213714e8ede None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:23.511801 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 109/217] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:43:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:23.511801 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501390be-f0af-4c1c-a364-0213714e8ede None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:23.530287 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa654c1e-0c44-4f61-bccc-bb73ffc0eb78 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:23.563475 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aedc428-10be-4b53-a8f3-a244ae1c8e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:23.564370 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aedc428-10be-4b53-a8f3-a244ae1c8e5f None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:23.564644 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aedc428-10be-4b53-a8f3-a244ae1c8e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:23.564911 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aedc428-10be-4b53-a8f3-a244ae1c8e5f None admin] PATH_INFO: `/v3/users/560712a0e3f54bbfb07a744fe6429fce` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:23.578094 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aedc428-10be-4b53-a8f3-a244ae1c8e5f None admin] RBAC: Authorizing `identity:get_user(user_id=560712a0e3f54bbfb07a744fe6429fce)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:23.579588 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aedc428-10be-4b53-a8f3-a244ae1c8e5f None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:23.581255 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 109/218] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:43:23 2022] GET /identity/v3/users/560712a0e3f54bbfb07a744fe6429fce => generated 297 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:23.581988 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aedc428-10be-4b53-a8f3-a244ae1c8e5f None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:23.621288 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501390be-f0af-4c1c-a364-0213714e8ede None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:23.653401 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c47ddf6-cdef-4d70-b9aa-2f35f9865c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:23.654184 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c47ddf6-cdef-4d70-b9aa-2f35f9865c5b None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:23.654462 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c47ddf6-cdef-4d70-b9aa-2f35f9865c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:23.654730 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c47ddf6-cdef-4d70-b9aa-2f35f9865c5b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:23.660953 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:23.665709 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c47ddf6-cdef-4d70-b9aa-2f35f9865c5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:23.673359 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-5c47ddf6-cdef-4d70-b9aa-2f35f9865c5b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:43:23.673359 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:23.673359 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:23.673359 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:23.673359 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:23.673359 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:23.673359 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:23.673359 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:23.673359 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:23.673359 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:23.673359 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:23.673359 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:23.673359 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:23.673359 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:23.673359 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:23.673359 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:23.674242 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:23.674242 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:23.674242 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:23.674242 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:23.674242 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:23.674242 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:23.674242 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:23.674242 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:23.674242 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:23.674242 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:23.674242 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:23.674242 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:23.674242 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:23.674242 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:23.674242 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:23.674242 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:23.675012 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:23.675012 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:23.675012 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:23.675012 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:23.675012 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:23.675012 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:23.675012 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:23.675012 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:23.675012 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:23.675012 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:23.675012 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:43:23.675012 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:43:23.675012 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 110/219] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jun 23 01:43:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:43:23.675928 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d830cb-01d2-4b64-89b5-fdb2a404c16b None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:23.676182 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d830cb-01d2-4b64-89b5-fdb2a404c16b None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:23.676406 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d830cb-01d2-4b64-89b5-fdb2a404c16b None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:23.677759 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 111/220] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:23.682493 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d830cb-01d2-4b64-89b5-fdb2a404c16b None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:23.690106 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aedc428-10be-4b53-a8f3-a244ae1c8e5f None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:23.725983 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb75081-168b-4a9f-899d-01ef39942a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:23.726676 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb75081-168b-4a9f-899d-01ef39942a13 None admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:23.726958 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb75081-168b-4a9f-899d-01ef39942a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:23.727050 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:23.727261 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb75081-168b-4a9f-899d-01ef39942a13 None admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:23.728374 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb75081-168b-4a9f-899d-01ef39942a13 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:23.729921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb75081-168b-4a9f-899d-01ef39942a13 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:23.750890 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 110/221] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Jun 23 01:43:23 2022] POST /identity/v3/users => generated 297 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:43:23.751967 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb75081-168b-4a9f-899d-01ef39942a13 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:23.795202 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d830cb-01d2-4b64-89b5-fdb2a404c16b None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:23.835282 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ba6f09-43e5-480a-9b64-035c955d5944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:23.836172 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba6f09-43e5-480a-9b64-035c955d5944 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:23.836814 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba6f09-43e5-480a-9b64-035c955d5944 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:23.836814 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba6f09-43e5-480a-9b64-035c955d5944 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:23.875096 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb75081-168b-4a9f-899d-01ef39942a13 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:23.909635 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:23.911228 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ba6f09-43e5-480a-9b64-035c955d5944 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:23.912870 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ba6f09-43e5-480a-9b64-035c955d5944 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:23.917516 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e188568-67a2-4114-9ba3-7765750249df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:23.918624 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e188568-67a2-4114-9ba3-7765750249df None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:23.918624 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e188568-67a2-4114-9ba3-7765750249df None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:23.918809 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e188568-67a2-4114-9ba3-7765750249df None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:23.920414 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e188568-67a2-4114-9ba3-7765750249df None admin] RBAC: Adding query filter params (name=admin) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:23.920414 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e188568-67a2-4114-9ba3-7765750249df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:23.922787 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e188568-67a2-4114-9ba3-7765750249df None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:23.932491 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 111/222] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:43:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:23.933895 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f9b77-06bc-4450-b28d-b7a7350d1ad3 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:23.934180 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f9b77-06bc-4450-b28d-b7a7350d1ad3 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:23.934451 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f9b77-06bc-4450-b28d-b7a7350d1ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:23.950875 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-c90f9b77-06bc-4450-b28d-b7a7350d1ad3 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:23.952085 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-c90f9b77-06bc-4450-b28d-b7a7350d1ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:23.968182 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:24.035967 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 112/223] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:43:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 01:43:24.036989 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ba6f09-43e5-480a-9b64-035c955d5944 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:24.047979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 112/224] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:24.049432 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f9b77-06bc-4450-b28d-b7a7350d1ad3 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:24.164787 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90f9b77-06bc-4450-b28d-b7a7350d1ad3 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:24.167236 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ba6f09-43e5-480a-9b64-035c955d5944 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:24.206170 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:24.219245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e33a1f7-5d9b-441f-9178-7d29708da738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:24.219245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33a1f7-5d9b-441f-9178-7d29708da738 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:24.219551 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fca5be-8bf2-4cd1-a126-786a7cfaaeca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:24.219639 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33a1f7-5d9b-441f-9178-7d29708da738 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:24.219639 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33a1f7-5d9b-441f-9178-7d29708da738 None admin] PATH_INFO: `/v3/users/f8870aeb21ef413b80b1c98c50efba39` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:24.219783 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fca5be-8bf2-4cd1-a126-786a7cfaaeca None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:24.220163 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fca5be-8bf2-4cd1-a126-786a7cfaaeca None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:24.220282 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e33a1f7-5d9b-441f-9178-7d29708da738 None admin] RBAC: Authorizing `identity:get_user(user_id=f8870aeb21ef413b80b1c98c50efba39)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:24.223233 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fca5be-8bf2-4cd1-a126-786a7cfaaeca None admin] PATH_INFO: `/v3/projects/9db43d2e143f49efbddf812fd9d03c69` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:24.223233 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fca5be-8bf2-4cd1-a126-786a7cfaaeca None admin] RBAC: Authorizing `identity:get_project(project_id=9db43d2e143f49efbddf812fd9d03c69)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:24.223429 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e33a1f7-5d9b-441f-9178-7d29708da738 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:24.223501 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fca5be-8bf2-4cd1-a126-786a7cfaaeca None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:24.223575 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 113/225] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:43:24 2022] GET /identity/v3/users/f8870aeb21ef413b80b1c98c50efba39 => generated 283 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:24.224236 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e33a1f7-5d9b-441f-9178-7d29708da738 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:24.229840 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 113/226] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:43:24 2022] GET /identity/v3/projects/9db43d2e143f49efbddf812fd9d03c69 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:24.231317 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fca5be-8bf2-4cd1-a126-786a7cfaaeca None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:24.278413 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:24.385731 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e33a1f7-5d9b-441f-9178-7d29708da738 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:24.385925 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fca5be-8bf2-4cd1-a126-786a7cfaaeca None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:24.431711 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa23a47-0052-4de7-a42f-c478f9dd59db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:24.432969 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa23a47-0052-4de7-a42f-c478f9dd59db None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:24.433464 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa23a47-0052-4de7-a42f-c478f9dd59db None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:24.433970 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa23a47-0052-4de7-a42f-c478f9dd59db None admin] PATH_INFO: `/v3/projects/9db43d2e143f49efbddf812fd9d03c69` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:24.435387 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa23a47-0052-4de7-a42f-c478f9dd59db None admin] RBAC: Authorizing `identity:get_project(project_id=9db43d2e143f49efbddf812fd9d03c69)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:24.437031 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa23a47-0052-4de7-a42f-c478f9dd59db None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:24.438920 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 114/227] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:43:24 2022] GET /identity/v3/projects/9db43d2e143f49efbddf812fd9d03c69 => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:24.440150 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa23a47-0052-4de7-a42f-c478f9dd59db None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:24.461249 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe2506-1305-4ba8-a2e8-dc8580b7416e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:24.463577 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe2506-1305-4ba8-a2e8-dc8580b7416e None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:24.464861 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe2506-1305-4ba8-a2e8-dc8580b7416e None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:24.465358 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe2506-1305-4ba8-a2e8-dc8580b7416e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:24.483287 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fe2506-1305-4ba8-a2e8-dc8580b7416e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:24.498088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-e0fe2506-1305-4ba8-a2e8-dc8580b7416e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 01:43:24.498088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:24.498088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:24.498088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:24.498088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:24.498088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:24.498088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:24.498088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:24.498088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:24.498088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:24.498088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:24.498088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:24.498088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:24.498088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:24.498088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:24.498088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:24.498976 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:24.498976 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:24.498976 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:24.498976 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:24.498976 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:24.498976 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:24.498976 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:24.498976 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:24.498976 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:24.498976 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:24.498976 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:24.498976 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:24.498976 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:24.498976 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:24.498976 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:24.498976 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:24.499781 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:24.499781 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:24.499781 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:24.499781 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:24.499781 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:24.499781 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:24.499781 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:24.499781 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:24.499781 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:24.499781 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:24.499781 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 01:43:24.499781 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:43:24.503906 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 114/228] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Jun 23 01:43:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:43:24.506185 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe2506-1305-4ba8-a2e8-dc8580b7416e None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:24.547582 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:24.606838 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa23a47-0052-4de7-a42f-c478f9dd59db None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:24.635135 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:24.646009 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e09aff-6954-479e-b3a9-767dbaeea38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:24.648118 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e09aff-6954-479e-b3a9-767dbaeea38b None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:24.648647 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e09aff-6954-479e-b3a9-767dbaeea38b None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:24.649143 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e09aff-6954-479e-b3a9-767dbaeea38b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:24.650340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e09aff-6954-479e-b3a9-767dbaeea38b None admin] RBAC: Adding query filter params (role.id=9021e3c4e3384a309951be4cc1d852e5, scope.project.id=9db43d2e143f49efbddf812fd9d03c69, user.id=560712a0e3f54bbfb07a744fe6429fce) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:24.650853 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e09aff-6954-479e-b3a9-767dbaeea38b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:24.652672 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e09aff-6954-479e-b3a9-767dbaeea38b None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:24.659145 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 115/229] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:43:24 2022] GET /identity/v3/role_assignments?user.id=560712a0e3f54bbfb07a744fe6429fce&scope.project.id=9db43d2e143f49efbddf812fd9d03c69&role.id=9021e3c4e3384a309951be4cc1d852e5 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:24.660611 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e09aff-6954-479e-b3a9-767dbaeea38b None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:24.713909 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe2506-1305-4ba8-a2e8-dc8580b7416e None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:24.812987 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e09aff-6954-479e-b3a9-767dbaeea38b None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:24.825339 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa80d6f8-b50f-4544-9f68-de4ff0a0caee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:24.826976 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80d6f8-b50f-4544-9f68-de4ff0a0caee None admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:24.827526 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80d6f8-b50f-4544-9f68-de4ff0a0caee None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:24.828386 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa80d6f8-b50f-4544-9f68-de4ff0a0caee None admin] PATH_INFO: `/v3/projects/9db43d2e143f49efbddf812fd9d03c69/users/f8870aeb21ef413b80b1c98c50efba39/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:24.834936 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa80d6f8-b50f-4544-9f68-de4ff0a0caee None admin] RBAC: Authorizing `identity:create_grant(project_id=9db43d2e143f49efbddf812fd9d03c69, user_id=f8870aeb21ef413b80b1c98c50efba39, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:24.837348 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa80d6f8-b50f-4544-9f68-de4ff0a0caee None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:24.839315 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:24.857360 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af9a9c8-3b7d-4080-9d54-34cf3a26faec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:24.858393 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9a9c8-3b7d-4080-9d54-34cf3a26faec None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:24.858866 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9a9c8-3b7d-4080-9d54-34cf3a26faec None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:24.859358 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9a9c8-3b7d-4080-9d54-34cf3a26faec None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:24.860652 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af9a9c8-3b7d-4080-9d54-34cf3a26faec None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:24.863647 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af9a9c8-3b7d-4080-9d54-34cf3a26faec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:24.865655 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af9a9c8-3b7d-4080-9d54-34cf3a26faec None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:24.885691 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 115/230] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jun 23 01:43:24 2022] PUT /identity/v3/projects/9db43d2e143f49efbddf812fd9d03c69/users/f8870aeb21ef413b80b1c98c50efba39/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:43:24.897261 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 116/231] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:43:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:24.903377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa80d6f8-b50f-4544-9f68-de4ff0a0caee None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:25.042547 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa80d6f8-b50f-4544-9f68-de4ff0a0caee None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:25.086558 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6cd621-364e-465e-b55c-e4367788bbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:25.087463 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6cd621-364e-465e-b55c-e4367788bbac None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:25.087748 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6cd621-364e-465e-b55c-e4367788bbac None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:25.088024 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6cd621-364e-465e-b55c-e4367788bbac None admin] PATH_INFO: `/v3/users/f8870aeb21ef413b80b1c98c50efba39` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:25.089239 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6cd621-364e-465e-b55c-e4367788bbac None admin] RBAC: Authorizing `identity:get_user(user_id=f8870aeb21ef413b80b1c98c50efba39)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:25.090711 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6cd621-364e-465e-b55c-e4367788bbac None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:25.092383 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 116/232] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:43:24 2022] GET /identity/v3/users/f8870aeb21ef413b80b1c98c50efba39 => generated 283 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:25.099738 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af9a9c8-3b7d-4080-9d54-34cf3a26faec None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:25.217409 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af9a9c8-3b7d-4080-9d54-34cf3a26faec None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:25.258257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3d92d3-6e22-4927-bd84-09243e758906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:25.260027 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d92d3-6e22-4927-bd84-09243e758906 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:25.260501 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d92d3-6e22-4927-bd84-09243e758906 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:25.260961 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d92d3-6e22-4927-bd84-09243e758906 None admin] PATH_INFO: `/v3/projects/9db43d2e143f49efbddf812fd9d03c69` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:25.268052 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3d92d3-6e22-4927-bd84-09243e758906 None admin] RBAC: Authorizing `identity:get_project(project_id=9db43d2e143f49efbddf812fd9d03c69)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:25.269773 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3d92d3-6e22-4927-bd84-09243e758906 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:25.272178 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 117/233] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:43:25 2022] GET /identity/v3/projects/9db43d2e143f49efbddf812fd9d03c69 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:25.277460 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6cd621-364e-465e-b55c-e4367788bbac None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:25.283029 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:25.433189 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6cd621-364e-465e-b55c-e4367788bbac None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:25.482622 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d8fd57-ab66-4633-87d6-2cf7e21c3741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:25.483603 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d8fd57-ab66-4633-87d6-2cf7e21c3741 None admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:25.483880 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d8fd57-ab66-4633-87d6-2cf7e21c3741 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:25.484148 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d8fd57-ab66-4633-87d6-2cf7e21c3741 None admin] PATH_INFO: `/v3/projects/9db43d2e143f49efbddf812fd9d03c69/users/f8870aeb21ef413b80b1c98c50efba39/roles/1c7adeca27524e258f7ddd2c107e43e1` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:25.486472 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d8fd57-ab66-4633-87d6-2cf7e21c3741 None admin] RBAC: Authorizing `identity:create_grant(project_id=9db43d2e143f49efbddf812fd9d03c69, user_id=f8870aeb21ef413b80b1c98c50efba39, role_id=1c7adeca27524e258f7ddd2c107e43e1)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:25.487972 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d8fd57-ab66-4633-87d6-2cf7e21c3741 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:25.517215 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 117/234] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jun 23 01:43:25 2022] PUT /identity/v3/projects/9db43d2e143f49efbddf812fd9d03c69/users/f8870aeb21ef413b80b1c98c50efba39/roles/1c7adeca27524e258f7ddd2c107e43e1 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:43:26.141273 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc34ce-e96e-4073-a963-f298823c5a7f None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:26.141273 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc34ce-e96e-4073-a963-f298823c5a7f None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:26.141273 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc34ce-e96e-4073-a963-f298823c5a7f None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:26.141273 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 118/235] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:26.153722 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23e616-d561-4ec7-9540-d8b75a1ab686 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:26.154641 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23e616-d561-4ec7-9540-d8b75a1ab686 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:26.155104 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23e616-d561-4ec7-9540-d8b75a1ab686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:26.184926 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-ae23e616-d561-4ec7-9540-d8b75a1ab686 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:26.186442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-ae23e616-d561-4ec7-9540-d8b75a1ab686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:26.260972 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:26.271991 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 118/236] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:26.313483 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e393a-e312-4715-aa98-8b97e66bc7d9 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:26.313827 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e393a-e312-4715-aa98-8b97e66bc7d9 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:26.314095 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e393a-e312-4715-aa98-8b97e66bc7d9 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:26.315282 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae23e616-d561-4ec7-9540-d8b75a1ab686 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:26.315722 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 119/237] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:26.320992 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51a34d-23e7-410d-aea7-d0005a6a9520 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:26.321275 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51a34d-23e7-410d-aea7-d0005a6a9520 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:26.321555 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51a34d-23e7-410d-aea7-d0005a6a9520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:26.354744 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-be51a34d-23e7-410d-aea7-d0005a6a9520 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:26.355994 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-be51a34d-23e7-410d-aea7-d0005a6a9520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:26.420187 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae23e616-d561-4ec7-9540-d8b75a1ab686 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:26.461755 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d59df-8446-43ea-9c23-c924e7d11c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:26.462551 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d59df-8446-43ea-9c23-c924e7d11c89 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:26.462828 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d59df-8446-43ea-9c23-c924e7d11c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:26.463099 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d59df-8446-43ea-9c23-c924e7d11c89 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:26.476219 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:26.482822 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d59df-8446-43ea-9c23-c924e7d11c89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:26.490676 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-060d59df-8446-43ea-9c23-c924e7d11c89 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 01:43:26.490676 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:26.490676 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:26.490676 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:26.490676 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:26.490676 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:26.490676 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:26.490676 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:26.490676 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:26.490676 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:26.490676 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:26.490676 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:26.490676 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:26.490676 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:26.490676 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:26.490676 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:26.491566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:26.491566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:26.491566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:26.491566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:26.491566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:26.491566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:26.491566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:26.491566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:26.491566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:26.491566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:26.491566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:26.491566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:26.491566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:26.491566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:26.491566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:26.491566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:26.492376 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:26.492376 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:26.492376 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:26.492376 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:26.492376 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:26.492376 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:26.492376 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:26.492376 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:26.492376 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:26.492376 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:26.492376 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 01:43:26.492376 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:43:26.492376 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 119/238] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:43:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:43:26.500759 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d59df-8446-43ea-9c23-c924e7d11c89 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:26.518711 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 120/239] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 01:43:26.555431 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be51a34d-23e7-410d-aea7-d0005a6a9520 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:26.558821 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:26.609488 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d59df-8446-43ea-9c23-c924e7d11c89 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:26.645018 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f782efea-080e-43e9-a337-2210f77e4113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:26.646033 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f782efea-080e-43e9-a337-2210f77e4113 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:26.646542 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f782efea-080e-43e9-a337-2210f77e4113 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:26.647012 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f782efea-080e-43e9-a337-2210f77e4113 None admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:26.648982 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f782efea-080e-43e9-a337-2210f77e4113 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:26.650809 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f782efea-080e-43e9-a337-2210f77e4113 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:26.653205 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f782efea-080e-43e9-a337-2210f77e4113 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:26.671145 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 120/240] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jun 23 01:43:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:26.679969 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f782efea-080e-43e9-a337-2210f77e4113 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:26.710278 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be51a34d-23e7-410d-aea7-d0005a6a9520 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:26.747924 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f835090-9aa8-4a9a-af97-c59e6b75670f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:26.749047 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f835090-9aa8-4a9a-af97-c59e6b75670f None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:26.749551 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f835090-9aa8-4a9a-af97-c59e6b75670f None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:26.750011 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f835090-9aa8-4a9a-af97-c59e6b75670f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:26.751768 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f835090-9aa8-4a9a-af97-c59e6b75670f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:26.753421 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f835090-9aa8-4a9a-af97-c59e6b75670f None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:26.755605 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 121/241] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:43:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:43:26.760887 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f835090-9aa8-4a9a-af97-c59e6b75670f None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:26.832648 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f782efea-080e-43e9-a337-2210f77e4113 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:26.847475 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:26.868931 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb91fc0-2022-4216-bb5c-d400f4254dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:26.870212 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb91fc0-2022-4216-bb5c-d400f4254dd7 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:26.870705 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb91fc0-2022-4216-bb5c-d400f4254dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:26.871238 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb91fc0-2022-4216-bb5c-d400f4254dd7 None admin] PATH_INFO: `/v3/users/af8e2d3b374b48e398c5826f0704f0a3` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:26.885406 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb91fc0-2022-4216-bb5c-d400f4254dd7 None admin] RBAC: Authorizing `identity:get_user(user_id=af8e2d3b374b48e398c5826f0704f0a3)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:26.887280 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb91fc0-2022-4216-bb5c-d400f4254dd7 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:26.887556 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:26.890253 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 121/242] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:43:26 2022] GET /identity/v3/users/af8e2d3b374b48e398c5826f0704f0a3 => generated 297 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:26.897918 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb91fc0-2022-4216-bb5c-d400f4254dd7 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:26.983602 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f835090-9aa8-4a9a-af97-c59e6b75670f None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:27.052560 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb91fc0-2022-4216-bb5c-d400f4254dd7 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:27.063924 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe3ed28-cab8-4c4b-9570-fd72ef16438f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:27.064947 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe3ed28-cab8-4c4b-9570-fd72ef16438f None admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:27.065430 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe3ed28-cab8-4c4b-9570-fd72ef16438f None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:27.065922 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe3ed28-cab8-4c4b-9570-fd72ef16438f None admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:27.066837 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe3ed28-cab8-4c4b-9570-fd72ef16438f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:27.068597 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe3ed28-cab8-4c4b-9570-fd72ef16438f None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:27.087832 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e68bd-7502-4bfc-943b-6ba0a38b5d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:27.088560 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e68bd-7502-4bfc-943b-6ba0a38b5d4d None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:27.088836 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e68bd-7502-4bfc-943b-6ba0a38b5d4d None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:27.089105 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e68bd-7502-4bfc-943b-6ba0a38b5d4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:27.122490 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 122/243] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Jun 23 01:43:26 2022] POST /identity/v3/users => generated 293 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:43:27.133054 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:27.161312 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625e68bd-7502-4bfc-943b-6ba0a38b5d4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:27.162864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625e68bd-7502-4bfc-943b-6ba0a38b5d4d None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:27.270788 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 122/244] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:43:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 01:43:27.276347 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe3ed28-cab8-4c4b-9570-fd72ef16438f None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:27.360852 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60eab32-9f20-43e6-8c9a-5258bc83a19f None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:27.361482 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60eab32-9f20-43e6-8c9a-5258bc83a19f None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:27.361954 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60eab32-9f20-43e6-8c9a-5258bc83a19f None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:27.363705 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 123/245] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:27.373531 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad03d61-5341-4f08-92d9-79f9cc926c53 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:27.374078 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad03d61-5341-4f08-92d9-79f9cc926c53 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:27.374576 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad03d61-5341-4f08-92d9-79f9cc926c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:27.393189 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-3ad03d61-5341-4f08-92d9-79f9cc926c53 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:27.394690 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-3ad03d61-5341-4f08-92d9-79f9cc926c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:27.397596 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe3ed28-cab8-4c4b-9570-fd72ef16438f None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:27.408046 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:27.442743 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536a152c-26c8-4ba3-9c8f-85cec26c8bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:27.443951 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536a152c-26c8-4ba3-9c8f-85cec26c8bd7 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:27.444405 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536a152c-26c8-4ba3-9c8f-85cec26c8bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:27.444850 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536a152c-26c8-4ba3-9c8f-85cec26c8bd7 None admin] PATH_INFO: `/v3/projects/9db43d2e143f49efbddf812fd9d03c69` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:27.446895 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536a152c-26c8-4ba3-9c8f-85cec26c8bd7 None admin] RBAC: Authorizing `identity:get_project(project_id=9db43d2e143f49efbddf812fd9d03c69)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:27.448886 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536a152c-26c8-4ba3-9c8f-85cec26c8bd7 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:27.451876 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 123/246] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:43:27 2022] GET /identity/v3/projects/9db43d2e143f49efbddf812fd9d03c69 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:27.458543 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536a152c-26c8-4ba3-9c8f-85cec26c8bd7 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:27.478918 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 124/247] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 01:43:27.507952 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad03d61-5341-4f08-92d9-79f9cc926c53 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:27.510554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:27.580967 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536a152c-26c8-4ba3-9c8f-85cec26c8bd7 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:27.625915 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5423cf4-68c1-41d2-8f03-f7bc9755360e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:27.626924 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5423cf4-68c1-41d2-8f03-f7bc9755360e None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:27.627473 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5423cf4-68c1-41d2-8f03-f7bc9755360e None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:27.627911 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5423cf4-68c1-41d2-8f03-f7bc9755360e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:27.628918 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5423cf4-68c1-41d2-8f03-f7bc9755360e None admin] RBAC: Adding query filter params (role.id=d159e6edeef94e418a4616504e214063, scope.project.id=9db43d2e143f49efbddf812fd9d03c69, user.id=af8e2d3b374b48e398c5826f0704f0a3) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:27.629365 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5423cf4-68c1-41d2-8f03-f7bc9755360e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:27.631081 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5423cf4-68c1-41d2-8f03-f7bc9755360e None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:27.637459 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 124/248] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:43:27 2022] GET /identity/v3/role_assignments?user.id=af8e2d3b374b48e398c5826f0704f0a3&scope.project.id=9db43d2e143f49efbddf812fd9d03c69&role.id=d159e6edeef94e418a4616504e214063 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:43:27.638600 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aab4b2-9b8a-4dcc-a71c-e9c588db99fc None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:27.638877 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aab4b2-9b8a-4dcc-a71c-e9c588db99fc None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:27.639156 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aab4b2-9b8a-4dcc-a71c-e9c588db99fc None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:27.640728 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 125/249] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:27.648539 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354293c4-e44b-4808-b3a5-c24eece1018c None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:27.648920 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354293c4-e44b-4808-b3a5-c24eece1018c None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:27.649261 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354293c4-e44b-4808-b3a5-c24eece1018c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:27.663707 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad03d61-5341-4f08-92d9-79f9cc926c53 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:27.679657 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-354293c4-e44b-4808-b3a5-c24eece1018c None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:27.681878 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-354293c4-e44b-4808-b3a5-c24eece1018c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:27.706396 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e90524-d561-4317-bacc-434bbc0d215e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:27.707520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e90524-d561-4317-bacc-434bbc0d215e None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:27.707936 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e90524-d561-4317-bacc-434bbc0d215e None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:27.708419 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e90524-d561-4317-bacc-434bbc0d215e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:27.721411 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e90524-d561-4317-bacc-434bbc0d215e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:27.730035 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-b8e90524-d561-4317-bacc-434bbc0d215e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 01:43:27.730035 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:27.730035 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:27.730035 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:27.730035 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:27.730035 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:27.730035 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:27.730035 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:27.730035 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:27.730035 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:27.730035 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:27.730035 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:27.730035 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:27.730035 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:27.730035 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:27.730035 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:27.731166 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:27.731166 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:27.731166 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:27.731166 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:27.731166 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:27.731166 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:27.731166 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:27.731166 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:27.731166 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:27.731166 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:27.731166 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:27.731166 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:27.731166 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:27.731166 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:27.731166 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:27.731166 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:27.732258 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:27.732258 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:27.732258 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:27.732258 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:27.732258 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:27.732258 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:27.732258 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:27.732258 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:27.732258 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:27.732258 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:27.732258 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 01:43:27.732258 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:43:27.733324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:27.734856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 125/250] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:43:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:43:27.741153 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e90524-d561-4317-bacc-434bbc0d215e None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:27.767001 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 126/251] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:27.808958 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354293c4-e44b-4808-b3a5-c24eece1018c None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:27.851016 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:27.904249 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e90524-d561-4317-bacc-434bbc0d215e None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:27.933282 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354293c4-e44b-4808-b3a5-c24eece1018c None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:27.954346 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b4dcb5-a812-4034-991c-37412ae843de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:27.955422 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4dcb5-a812-4034-991c-37412ae843de None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:27.956913 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4dcb5-a812-4034-991c-37412ae843de None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:27.957385 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4dcb5-a812-4034-991c-37412ae843de None admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:27.960022 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b4dcb5-a812-4034-991c-37412ae843de None admin] RBAC: Adding query filter params (name=member) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:27.960815 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b4dcb5-a812-4034-991c-37412ae843de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:27.963646 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b4dcb5-a812-4034-991c-37412ae843de None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:27.981811 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 126/252] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jun 23 01:43:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:43:27.987490 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b4dcb5-a812-4034-991c-37412ae843de None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:28.006336 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264da6d-37b1-4419-975b-771b56429e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:28.007998 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264da6d-37b1-4419-975b-771b56429e4c None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:28.008506 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264da6d-37b1-4419-975b-771b56429e4c None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:28.008973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264da6d-37b1-4419-975b-771b56429e4c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:28.022462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e264da6d-37b1-4419-975b-771b56429e4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:28.031345 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-e264da6d-37b1-4419-975b-771b56429e4c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:43:28.031345 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:28.031345 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:28.031345 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:28.031345 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:28.031345 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:28.031345 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:28.031345 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:28.031345 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:28.031345 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:28.031345 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:28.031345 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:28.031345 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:28.031345 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:28.031345 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:28.031345 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:28.032242 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:28.032242 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:28.032242 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:28.032242 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:28.032242 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:28.032242 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:28.032242 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:28.032242 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:28.032242 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:28.032242 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:28.032242 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:28.032242 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:28.032242 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:28.032242 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:28.032242 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:28.032242 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:28.033004 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:28.033004 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:28.033004 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:28.033004 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:28.033004 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:28.033004 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:28.033004 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:28.033004 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:28.033004 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:28.033004 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:28.033004 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:43:28.033004 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:43:28.036433 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 127/253] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jun 23 01:43:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:43:28.043027 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264da6d-37b1-4419-975b-771b56429e4c None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:28.078607 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:28.159258 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264da6d-37b1-4419-975b-771b56429e4c None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:28.210222 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aef77c9-7a0f-4129-8243-a9ba32523dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:28.210907 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef77c9-7a0f-4129-8243-a9ba32523dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:28.211191 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef77c9-7a0f-4129-8243-a9ba32523dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:28.211647 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef77c9-7a0f-4129-8243-a9ba32523dd1 None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:28.212413 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aef77c9-7a0f-4129-8243-a9ba32523dd1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:28.212920 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aef77c9-7a0f-4129-8243-a9ba32523dd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:28.213314 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b4dcb5-a812-4034-991c-37412ae843de None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:28.214455 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aef77c9-7a0f-4129-8243-a9ba32523dd1 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:28.224450 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 128/254] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:43:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:28.232695 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aef77c9-7a0f-4129-8243-a9ba32523dd1 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:28.273048 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:28.301294 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8619bb-e9ea-4573-8b76-591e41d872cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:28.302331 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8619bb-e9ea-4573-8b76-591e41d872cc None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:28.302609 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8619bb-e9ea-4573-8b76-591e41d872cc None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:28.302879 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8619bb-e9ea-4573-8b76-591e41d872cc None admin] PATH_INFO: `/v3/users/560712a0e3f54bbfb07a744fe6429fce` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:28.309676 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8619bb-e9ea-4573-8b76-591e41d872cc None admin] RBAC: Authorizing `identity:get_user(user_id=560712a0e3f54bbfb07a744fe6429fce)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:28.311232 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8619bb-e9ea-4573-8b76-591e41d872cc None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:28.313283 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 127/255] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:43:27 2022] GET /identity/v3/users/560712a0e3f54bbfb07a744fe6429fce => generated 297 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:28.314615 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998aea98-9231-4ce1-8fda-e81408202821 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:28.314903 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998aea98-9231-4ce1-8fda-e81408202821 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:28.315183 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998aea98-9231-4ce1-8fda-e81408202821 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:28.316616 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 128/256] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:28.321510 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998aea98-9231-4ce1-8fda-e81408202821 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:28.371575 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:28.386967 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aef77c9-7a0f-4129-8243-a9ba32523dd1 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:28.453763 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe39421b-1b30-4e07-a4fd-7fc527d9e6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:28.454664 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39421b-1b30-4e07-a4fd-7fc527d9e6be None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:28.454942 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39421b-1b30-4e07-a4fd-7fc527d9e6be None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:28.455239 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39421b-1b30-4e07-a4fd-7fc527d9e6be None admin] PATH_INFO: `/v3/users/f8870aeb21ef413b80b1c98c50efba39` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:28.456463 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe39421b-1b30-4e07-a4fd-7fc527d9e6be None admin] RBAC: Authorizing `identity:get_user(user_id=f8870aeb21ef413b80b1c98c50efba39)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:28.457935 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe39421b-1b30-4e07-a4fd-7fc527d9e6be None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:28.459881 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 129/257] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:43:28 2022] GET /identity/v3/users/f8870aeb21ef413b80b1c98c50efba39 => generated 283 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:28.461150 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0520c43-8d1f-4e87-a430-0a7f55048e0f None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:28.461690 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0520c43-8d1f-4e87-a430-0a7f55048e0f None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:28.461973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0520c43-8d1f-4e87-a430-0a7f55048e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:28.495280 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-d0520c43-8d1f-4e87-a430-0a7f55048e0f None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:28.496495 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-d0520c43-8d1f-4e87-a430-0a7f55048e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:28.526895 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998aea98-9231-4ce1-8fda-e81408202821 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:28.589229 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e7e73-d38e-48ab-9722-3286567b98c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:28.590099 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e7e73-d38e-48ab-9722-3286567b98c5 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:28.590380 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e7e73-d38e-48ab-9722-3286567b98c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:28.590652 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e7e73-d38e-48ab-9722-3286567b98c5 None admin] PATH_INFO: `/v3/projects/9db43d2e143f49efbddf812fd9d03c69` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:28.591902 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4e7e73-d38e-48ab-9722-3286567b98c5 None admin] RBAC: Authorizing `identity:get_project(project_id=9db43d2e143f49efbddf812fd9d03c69)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:28.595041 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4e7e73-d38e-48ab-9722-3286567b98c5 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:28.598003 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 129/258] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:43:28 2022] GET /identity/v3/projects/9db43d2e143f49efbddf812fd9d03c69 => generated 302 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:28.598823 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e7e73-d38e-48ab-9722-3286567b98c5 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:28.638914 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 130/259] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:28.641259 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0520c43-8d1f-4e87-a430-0a7f55048e0f None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:28.719178 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:28.797494 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e7e73-d38e-48ab-9722-3286567b98c5 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:28.803554 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:28.822043 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0520c43-8d1f-4e87-a430-0a7f55048e0f None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:28.861481 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07290227-0173-42b2-9662-a867086402bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:28.862131 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07290227-0173-42b2-9662-a867086402bb None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:28.862411 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07290227-0173-42b2-9662-a867086402bb None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:28.862685 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07290227-0173-42b2-9662-a867086402bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:28.885660 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab6598d-6755-4983-aeb0-cedc3073ca5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:28.886561 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6598d-6755-4983-aeb0-cedc3073ca5b None admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:28.886840 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6598d-6755-4983-aeb0-cedc3073ca5b None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:28.887110 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6598d-6755-4983-aeb0-cedc3073ca5b None admin] PATH_INFO: `/v3/projects/9db43d2e143f49efbddf812fd9d03c69/users/560712a0e3f54bbfb07a744fe6429fce/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:28.890579 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab6598d-6755-4983-aeb0-cedc3073ca5b None admin] RBAC: Authorizing `identity:create_grant(project_id=9db43d2e143f49efbddf812fd9d03c69, user_id=560712a0e3f54bbfb07a744fe6429fce, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:28.892064 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab6598d-6755-4983-aeb0-cedc3073ca5b None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:28.917730 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 131/260] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jun 23 01:43:28 2022] PUT /identity/v3/projects/9db43d2e143f49efbddf812fd9d03c69/users/560712a0e3f54bbfb07a744fe6429fce/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:43:28.918360 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab6598d-6755-4983-aeb0-cedc3073ca5b None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:28.990189 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07290227-0173-42b2-9662-a867086402bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:28.991632 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07290227-0173-42b2-9662-a867086402bb None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:29.036404 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab6598d-6755-4983-aeb0-cedc3073ca5b None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:29.074436 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c787046-016a-4f67-b8f7-ad25310cfc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:29.075670 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c787046-016a-4f67-b8f7-ad25310cfc2f None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:29.076137 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c787046-016a-4f67-b8f7-ad25310cfc2f None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:29.076590 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c787046-016a-4f67-b8f7-ad25310cfc2f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:29.088669 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c787046-016a-4f67-b8f7-ad25310cfc2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:29.097165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-3c787046-016a-4f67-b8f7-ad25310cfc2f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 01:43:29.097165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:29.097165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:29.097165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:29.097165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:29.097165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:29.097165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:29.097165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:29.097165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:29.097165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:29.097165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:29.097165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:29.097165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:29.097165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:29.097165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:29.097165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:29.098063 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:29.098063 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:29.098063 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:29.098063 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:29.098063 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:29.098063 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:29.098063 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:29.098063 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:29.098063 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:29.098063 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:29.098063 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:29.098063 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:29.098063 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:29.098063 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:29.098063 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:29.098063 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:29.098896 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:29.098896 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:29.098896 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:29.098896 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:29.098896 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:29.098896 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:29.098896 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:29.098896 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:29.098896 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:29.098896 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:29.098896 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 01:43:29.098896 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:43:29.101423 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 132/261] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Jun 23 01:43:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:43:29.108676 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c787046-016a-4f67-b8f7-ad25310cfc2f None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:29.121850 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:29.137341 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:29.224392 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 130/262] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:43:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 626 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 01:43:29.229865 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07290227-0173-42b2-9662-a867086402bb None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:29.286528 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c787046-016a-4f67-b8f7-ad25310cfc2f None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:29.339961 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0846d3-bf7c-43b9-a566-18de5d1c453a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:29.341332 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0846d3-bf7c-43b9-a566-18de5d1c453a None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:29.341806 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0846d3-bf7c-43b9-a566-18de5d1c453a None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:29.342310 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0846d3-bf7c-43b9-a566-18de5d1c453a None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:29.343321 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0846d3-bf7c-43b9-a566-18de5d1c453a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:29.343819 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0846d3-bf7c-43b9-a566-18de5d1c453a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:29.345428 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0846d3-bf7c-43b9-a566-18de5d1c453a None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:29.360205 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 133/263] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:43:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:29.367145 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0846d3-bf7c-43b9-a566-18de5d1c453a None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:29.410668 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07290227-0173-42b2-9662-a867086402bb None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:29.446617 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:29.480859 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0846d3-bf7c-43b9-a566-18de5d1c453a None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:29.508284 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c7079d-c2e9-45de-a64b-a62fea4f7c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:29.509541 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c7079d-c2e9-45de-a64b-a62fea4f7c8a None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:29.509823 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c7079d-c2e9-45de-a64b-a62fea4f7c8a None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:29.510103 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c7079d-c2e9-45de-a64b-a62fea4f7c8a None admin] PATH_INFO: `/v3/projects/9db43d2e143f49efbddf812fd9d03c69` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:29.511789 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c7079d-c2e9-45de-a64b-a62fea4f7c8a None admin] RBAC: Authorizing `identity:get_project(project_id=9db43d2e143f49efbddf812fd9d03c69)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:29.513676 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c7079d-c2e9-45de-a64b-a62fea4f7c8a None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:29.520496 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e4353-fd84-48a9-8296-cc64b025785d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:29.521700 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e4353-fd84-48a9-8296-cc64b025785d None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:29.522356 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 131/264] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:43:29 2022] GET /identity/v3/projects/9db43d2e143f49efbddf812fd9d03c69 => generated 302 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:43:29.522789 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e4353-fd84-48a9-8296-cc64b025785d None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:29.523377 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e4353-fd84-48a9-8296-cc64b025785d None admin] PATH_INFO: `/v3/users/f8870aeb21ef413b80b1c98c50efba39` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:29.524965 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7e4353-fd84-48a9-8296-cc64b025785d None admin] RBAC: Authorizing `identity:get_user(user_id=f8870aeb21ef413b80b1c98c50efba39)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:29.526780 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7e4353-fd84-48a9-8296-cc64b025785d None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:29.528174 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c7079d-c2e9-45de-a64b-a62fea4f7c8a None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:29.529076 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 134/265] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:43:29 2022] GET /identity/v3/users/f8870aeb21ef413b80b1c98c50efba39 => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:43:29.534734 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e4353-fd84-48a9-8296-cc64b025785d None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:29.628446 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:29.650673 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e4353-fd84-48a9-8296-cc64b025785d None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:29.689597 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f474a3f0-4a0e-4211-acf5-f1627e2e1e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:29.690616 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474a3f0-4a0e-4211-acf5-f1627e2e1e87 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:29.691141 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474a3f0-4a0e-4211-acf5-f1627e2e1e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:29.691758 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474a3f0-4a0e-4211-acf5-f1627e2e1e87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:29.719788 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:29.822245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c7079d-c2e9-45de-a64b-a62fea4f7c8a None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:29.850149 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f474a3f0-4a0e-4211-acf5-f1627e2e1e87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:29.851881 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f474a3f0-4a0e-4211-acf5-f1627e2e1e87 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:29.880928 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6256cd6c-5dc8-4ef0-9c84-655d62f17ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:29.882015 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256cd6c-5dc8-4ef0-9c84-655d62f17ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:29.882504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256cd6c-5dc8-4ef0-9c84-655d62f17ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:29.882960 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256cd6c-5dc8-4ef0-9c84-655d62f17ce0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:29.884042 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6256cd6c-5dc8-4ef0-9c84-655d62f17ce0 None admin] RBAC: Adding query filter params (role.id=7b20d65f49ee4a48894b6c060bb5d580, scope.project.id=9db43d2e143f49efbddf812fd9d03c69, user.id=f8870aeb21ef413b80b1c98c50efba39) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:29.884542 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6256cd6c-5dc8-4ef0-9c84-655d62f17ce0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:29.886195 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6256cd6c-5dc8-4ef0-9c84-655d62f17ce0 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:29.893682 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 132/266] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:43:29 2022] GET /identity/v3/role_assignments?user.id=f8870aeb21ef413b80b1c98c50efba39&scope.project.id=9db43d2e143f49efbddf812fd9d03c69&role.id=7b20d65f49ee4a48894b6c060bb5d580 => generated 608 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:30.028339 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 135/267] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:43:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 01:43:30.034330 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6256cd6c-5dc8-4ef0-9c84-655d62f17ce0 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:30.173001 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:30.182671 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6256cd6c-5dc8-4ef0-9c84-655d62f17ce0 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:30.232294 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa89976-3a5a-47e3-a1c2-99713fd01b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:30.233162 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa89976-3a5a-47e3-a1c2-99713fd01b2b None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:30.233447 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa89976-3a5a-47e3-a1c2-99713fd01b2b None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:30.233724 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa89976-3a5a-47e3-a1c2-99713fd01b2b None admin] PATH_INFO: `/v3/projects/9db43d2e143f49efbddf812fd9d03c69` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:30.234959 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa89976-3a5a-47e3-a1c2-99713fd01b2b None admin] RBAC: Authorizing `identity:get_project(project_id=9db43d2e143f49efbddf812fd9d03c69)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:30.236482 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa89976-3a5a-47e3-a1c2-99713fd01b2b None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:30.238496 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 133/268] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:43:30 2022] GET /identity/v3/projects/9db43d2e143f49efbddf812fd9d03c69 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:30.245385 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f474a3f0-4a0e-4211-acf5-f1627e2e1e87 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:30.310410 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43893272-997f-4ba2-875b-b7501de4062c None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:30.310706 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43893272-997f-4ba2-875b-b7501de4062c None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:30.310974 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43893272-997f-4ba2-875b-b7501de4062c None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:30.312698 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 134/269] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:30.318151 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e96cf-aaad-473c-80ab-56fb901ff496 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:30.318442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e96cf-aaad-473c-80ab-56fb901ff496 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:30.318721 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e96cf-aaad-473c-80ab-56fb901ff496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:30.331461 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:30.352221 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-137e96cf-aaad-473c-80ab-56fb901ff496 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:30.353427 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-137e96cf-aaad-473c-80ab-56fb901ff496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:30.401844 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f474a3f0-4a0e-4211-acf5-f1627e2e1e87 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:30.445237 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c844ab3-7f00-4394-8660-ca0e3dda459f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:30.445911 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c844ab3-7f00-4394-8660-ca0e3dda459f None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:30.447101 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c844ab3-7f00-4394-8660-ca0e3dda459f None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:30.447101 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c844ab3-7f00-4394-8660-ca0e3dda459f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:30.447735 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c844ab3-7f00-4394-8660-ca0e3dda459f None admin] RBAC: Adding query filter params (role.id=1c7adeca27524e258f7ddd2c107e43e1, scope.project.id=9db43d2e143f49efbddf812fd9d03c69, user.id=f8870aeb21ef413b80b1c98c50efba39) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:30.447735 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c844ab3-7f00-4394-8660-ca0e3dda459f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:30.449120 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c844ab3-7f00-4394-8660-ca0e3dda459f None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:30.455806 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 136/270] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:43:30 2022] GET /identity/v3/role_assignments?user.id=f8870aeb21ef413b80b1c98c50efba39&scope.project.id=9db43d2e143f49efbddf812fd9d03c69&role.id=1c7adeca27524e258f7ddd2c107e43e1 => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:30.457105 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433bd576-0918-4926-b737-47f8f47eefcc None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:30.457420 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433bd576-0918-4926-b737-47f8f47eefcc None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:30.457657 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433bd576-0918-4926-b737-47f8f47eefcc None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:30.458970 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 137/271] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:30.460746 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0263e9e-1bba-4193-8ba3-76ddaa77d4f8 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:30.461030 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0263e9e-1bba-4193-8ba3-76ddaa77d4f8 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:30.461308 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0263e9e-1bba-4193-8ba3-76ddaa77d4f8 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:30.462650 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 138/272] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:30.468205 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3752571-40fc-4a08-81d1-f7bf531798fb None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:30.468507 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3752571-40fc-4a08-81d1-f7bf531798fb None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:30.468763 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3752571-40fc-4a08-81d1-f7bf531798fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:30.476068 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 135/273] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 01:43:30.477819 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1861959-ce5e-4a9e-b8eb-f00915632595 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:30.478303 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1861959-ce5e-4a9e-b8eb-f00915632595 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:30.478792 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1861959-ce5e-4a9e-b8eb-f00915632595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:30.494720 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-a3752571-40fc-4a08-81d1-f7bf531798fb None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:30.496018 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-a3752571-40fc-4a08-81d1-f7bf531798fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:30.501247 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-e1861959-ce5e-4a9e-b8eb-f00915632595 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:30.504073 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-e1861959-ce5e-4a9e-b8eb-f00915632595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:30.585801 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 139/274] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:30.586764 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3752571-40fc-4a08-81d1-f7bf531798fb None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:30.593389 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 136/275] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:30.611844 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1861959-ce5e-4a9e-b8eb-f00915632595 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:30.640657 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:30.647250 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:30.724115 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3752571-40fc-4a08-81d1-f7bf531798fb None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:30.750866 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1861959-ce5e-4a9e-b8eb-f00915632595 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:30.762898 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55d6403-cacd-4758-933b-a145eda3a5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:30.763731 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d6403-cacd-4758-933b-a145eda3a5e4 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:30.764011 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d6403-cacd-4758-933b-a145eda3a5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:30.764282 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55d6403-cacd-4758-933b-a145eda3a5e4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:30.775998 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55d6403-cacd-4758-933b-a145eda3a5e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:30.783659 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-e55d6403-cacd-4758-933b-a145eda3a5e4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 01:43:30.783659 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:30.783659 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:30.783659 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:30.783659 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:30.783659 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:30.783659 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:30.783659 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:30.783659 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:30.783659 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:30.783659 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:30.783659 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:30.783659 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:30.783659 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:30.783659 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:30.783659 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:30.784540 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:30.784540 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:30.784540 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:30.784540 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:30.784540 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:30.784540 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:30.784540 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:30.784540 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:30.784540 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:30.784540 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:30.784540 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:30.784540 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:30.784540 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:30.784540 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:30.784540 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:30.784540 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:30.785327 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:30.785327 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:30.785327 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:30.785327 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:30.785327 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:30.785327 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:30.785327 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:30.785327 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:30.785327 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:30.785327 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:30.785327 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 01:43:30.785327 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:43:30.785327 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 140/276] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:43:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:43:30.786637 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55d6403-cacd-4758-933b-a145eda3a5e4 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:30.791230 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e9016-3dd4-4136-88a0-25203b7901a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:30.792126 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e9016-3dd4-4136-88a0-25203b7901a4 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:30.792473 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e9016-3dd4-4136-88a0-25203b7901a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:30.792812 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e9016-3dd4-4136-88a0-25203b7901a4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:30.794084 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2e9016-3dd4-4136-88a0-25203b7901a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:30.795664 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2e9016-3dd4-4136-88a0-25203b7901a4 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:30.797487 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 137/277] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:43:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:43:30.798189 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e9016-3dd4-4136-88a0-25203b7901a4 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:30.893124 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:30.893715 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:30.900921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55d6403-cacd-4758-933b-a145eda3a5e4 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:30.907883 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e9016-3dd4-4136-88a0-25203b7901a4 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:30.938317 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7067f3cd-d078-431d-b3d5-8bbb643c99e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:30.939120 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067f3cd-d078-431d-b3d5-8bbb643c99e9 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:30.939441 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067f3cd-d078-431d-b3d5-8bbb643c99e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:30.939716 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067f3cd-d078-431d-b3d5-8bbb643c99e9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:30.944816 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7595b3a6-dfc2-412a-b534-444e4d06f33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:30.945848 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595b3a6-dfc2-412a-b534-444e4d06f33c None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:30.946410 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595b3a6-dfc2-412a-b534-444e4d06f33c None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:30.946860 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595b3a6-dfc2-412a-b534-444e4d06f33c None admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:30.947872 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7595b3a6-dfc2-412a-b534-444e4d06f33c None admin] RBAC: Adding query filter params (name=member) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:30.948371 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7595b3a6-dfc2-412a-b534-444e4d06f33c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:30.949709 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7067f3cd-d078-431d-b3d5-8bbb643c99e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:30.950138 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7595b3a6-dfc2-412a-b534-444e4d06f33c None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:30.957358 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-7067f3cd-d078-431d-b3d5-8bbb643c99e9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 01:43:30.957358 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:30.957358 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:30.957358 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:30.957358 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:30.957358 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:30.957358 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:30.957358 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:30.957358 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:30.957358 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:30.957358 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:30.957358 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:30.957358 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:30.957358 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:30.957358 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:30.957358 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:30.958361 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:30.958361 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:30.958361 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:30.958361 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:30.958361 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:30.958361 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:30.958361 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:30.958361 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:30.958361 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:30.958361 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:30.958361 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:30.958361 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:30.958361 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:30.958361 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:30.958361 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:30.958361 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:30.959156 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:30.959156 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:30.959156 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:30.959156 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:30.959156 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:30.959156 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:30.959156 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:30.959156 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:30.959156 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:30.959156 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:30.959156 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 01:43:30.959156 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:43:30.959156 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 141/278] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:43:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:43:30.959869 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 138/279] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jun 23 01:43:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:30.964696 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7067f3cd-d078-431d-b3d5-8bbb643c99e9 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:30.964824 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7595b3a6-dfc2-412a-b534-444e4d06f33c None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:31.066011 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7067f3cd-d078-431d-b3d5-8bbb643c99e9 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:31.074401 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7595b3a6-dfc2-412a-b534-444e4d06f33c None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:31.099753 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f7be7-fc26-497b-91c5-9b001e8b5311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:31.100423 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f7be7-fc26-497b-91c5-9b001e8b5311 None admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:31.100704 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f7be7-fc26-497b-91c5-9b001e8b5311 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:31.100979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f7be7-fc26-497b-91c5-9b001e8b5311 None admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:31.101670 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4f7be7-fc26-497b-91c5-9b001e8b5311 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:31.103136 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4f7be7-fc26-497b-91c5-9b001e8b5311 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:31.108668 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ddff2-4f29-48f9-81ed-cf86e85b8e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:31.109491 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ddff2-4f29-48f9-81ed-cf86e85b8e01 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:31.109817 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ddff2-4f29-48f9-81ed-cf86e85b8e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:31.110128 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ddff2-4f29-48f9-81ed-cf86e85b8e01 None admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:31.110901 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4ddff2-4f29-48f9-81ed-cf86e85b8e01 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:31.111228 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4ddff2-4f29-48f9-81ed-cf86e85b8e01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:31.112651 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4ddff2-4f29-48f9-81ed-cf86e85b8e01 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:31.122984 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 139/280] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jun 23 01:43:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:43:31.123850 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ddff2-4f29-48f9-81ed-cf86e85b8e01 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:31.139250 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 142/281] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Jun 23 01:43:30 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:43:31.139250 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f7be7-fc26-497b-91c5-9b001e8b5311 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:31.148047 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:31.160362 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:31.255376 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f7be7-fc26-497b-91c5-9b001e8b5311 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:31.265251 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ddff2-4f29-48f9-81ed-cf86e85b8e01 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:31.305942 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca0aa9f-347d-4c3b-94bd-481e5f580cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:31.306834 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0aa9f-347d-4c3b-94bd-481e5f580cdf None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:31.307118 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0aa9f-347d-4c3b-94bd-481e5f580cdf None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:31.307416 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0aa9f-347d-4c3b-94bd-481e5f580cdf None admin] PATH_INFO: `/v3/users/af8e2d3b374b48e398c5826f0704f0a3` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:31.309390 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca0aa9f-347d-4c3b-94bd-481e5f580cdf None admin] RBAC: Authorizing `identity:get_user(user_id=af8e2d3b374b48e398c5826f0704f0a3)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:31.311023 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec68f5-1111-4ed3-b6d1-3536ca8c43c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:31.311157 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca0aa9f-347d-4c3b-94bd-481e5f580cdf None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:31.311935 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec68f5-1111-4ed3-b6d1-3536ca8c43c1 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:31.312217 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec68f5-1111-4ed3-b6d1-3536ca8c43c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:31.312493 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec68f5-1111-4ed3-b6d1-3536ca8c43c1 None admin] PATH_INFO: `/v3/users/f2ea425f1e674bb09fe0041dddccf44d` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:31.313451 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 143/282] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:43:31 2022] GET /identity/v3/users/af8e2d3b374b48e398c5826f0704f0a3 => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:31.318840 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca0aa9f-347d-4c3b-94bd-481e5f580cdf None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:31.332888 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ec68f5-1111-4ed3-b6d1-3536ca8c43c1 None admin] RBAC: Authorizing `identity:get_user(user_id=f2ea425f1e674bb09fe0041dddccf44d)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:31.334700 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ec68f5-1111-4ed3-b6d1-3536ca8c43c1 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:31.336671 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 140/283] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:43:31 2022] GET /identity/v3/users/f2ea425f1e674bb09fe0041dddccf44d => generated 293 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:31.342615 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec68f5-1111-4ed3-b6d1-3536ca8c43c1 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:31.423124 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:31.423888 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:31.459811 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca0aa9f-347d-4c3b-94bd-481e5f580cdf None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:31.460723 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec68f5-1111-4ed3-b6d1-3536ca8c43c1 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:31.501635 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f263d8f-8843-44c1-b922-0d5ce0217174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:31.501837 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9957d27c-d178-483c-86fc-ff01e1135363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:31.502310 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f263d8f-8843-44c1-b922-0d5ce0217174 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:31.502522 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957d27c-d178-483c-86fc-ff01e1135363 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:31.502616 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f263d8f-8843-44c1-b922-0d5ce0217174 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:31.502802 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957d27c-d178-483c-86fc-ff01e1135363 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:31.502899 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f263d8f-8843-44c1-b922-0d5ce0217174 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:31.503077 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957d27c-d178-483c-86fc-ff01e1135363 None admin] PATH_INFO: `/v3/projects/9db43d2e143f49efbddf812fd9d03c69` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:31.504116 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f263d8f-8843-44c1-b922-0d5ce0217174 None admin] RBAC: Adding query filter params (role.id=9021e3c4e3384a309951be4cc1d852e5, user.id=f2ea425f1e674bb09fe0041dddccf44d, scope.system=all) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:31.504116 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f263d8f-8843-44c1-b922-0d5ce0217174 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:31.504350 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9957d27c-d178-483c-86fc-ff01e1135363 None admin] RBAC: Authorizing `identity:get_project(project_id=9db43d2e143f49efbddf812fd9d03c69)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:31.505591 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f263d8f-8843-44c1-b922-0d5ce0217174 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:31.505830 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9957d27c-d178-483c-86fc-ff01e1135363 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:31.507705 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 144/284] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:43:31 2022] GET /identity/v3/projects/9db43d2e143f49efbddf812fd9d03c69 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:31.511991 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 141/285] 10.222.0.26 () {56 vars in 1308 bytes} [Thu Jun 23 01:43:31 2022] GET /identity/v3/role_assignments?user.id=f2ea425f1e674bb09fe0041dddccf44d&scope.system=all&role.id=9021e3c4e3384a309951be4cc1d852e5 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:31.513460 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9957d27c-d178-483c-86fc-ff01e1135363 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:31.626815 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9957d27c-d178-483c-86fc-ff01e1135363 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:31.664813 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e1fbfd-d5ac-4982-8990-c1898e978910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:31.665733 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1fbfd-d5ac-4982-8990-c1898e978910 None admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:31.666012 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1fbfd-d5ac-4982-8990-c1898e978910 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:31.666289 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1fbfd-d5ac-4982-8990-c1898e978910 None admin] PATH_INFO: `/v3/projects/9db43d2e143f49efbddf812fd9d03c69/users/af8e2d3b374b48e398c5826f0704f0a3/roles/d159e6edeef94e418a4616504e214063` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:31.668395 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e1fbfd-d5ac-4982-8990-c1898e978910 None admin] RBAC: Authorizing `identity:create_grant(project_id=9db43d2e143f49efbddf812fd9d03c69, user_id=af8e2d3b374b48e398c5826f0704f0a3, role_id=d159e6edeef94e418a4616504e214063)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:31.669877 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e1fbfd-d5ac-4982-8990-c1898e978910 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:31.688120 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 145/286] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jun 23 01:43:31 2022] PUT /identity/v3/projects/9db43d2e143f49efbddf812fd9d03c69/users/af8e2d3b374b48e398c5826f0704f0a3/roles/d159e6edeef94e418a4616504e214063 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 01:43:31.815593 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c70d60-d31c-4858-810e-983d2fe7b909 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:31.815923 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c70d60-d31c-4858-810e-983d2fe7b909 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:31.816162 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c70d60-d31c-4858-810e-983d2fe7b909 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:31.818140 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 142/287] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:31.832801 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e5a6e8-d45c-4938-91d1-98189965bc1a None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:31.833149 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e5a6e8-d45c-4938-91d1-98189965bc1a None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:31.833401 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e5a6e8-d45c-4938-91d1-98189965bc1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:31.845573 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:31.857092 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-c8e5a6e8-d45c-4938-91d1-98189965bc1a None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:31.858912 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-c8e5a6e8-d45c-4938-91d1-98189965bc1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:31.941038 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 146/288] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:31.966652 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c70d60-d31c-4858-810e-983d2fe7b909 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:32.108860 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c70d60-d31c-4858-810e-983d2fe7b909 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:32.153269 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54a389-6823-4385-b27c-9c095a525805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:32.154506 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54a389-6823-4385-b27c-9c095a525805 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:32.155007 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54a389-6823-4385-b27c-9c095a525805 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:32.155517 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54a389-6823-4385-b27c-9c095a525805 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:32.157626 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:32.170990 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b54a389-6823-4385-b27c-9c095a525805 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:32.179316 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-8b54a389-6823-4385-b27c-9c095a525805 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 01:43:32.179316 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:32.179316 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:32.179316 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:32.179316 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:32.179316 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:32.179316 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:32.179316 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:32.179316 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:32.179316 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:32.179316 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:32.179316 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:32.179316 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:32.179316 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:32.179316 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:32.179316 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:32.180184 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:32.180184 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:32.180184 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:32.180184 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:32.180184 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:32.180184 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:32.180184 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:32.180184 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:32.180184 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:32.180184 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:32.180184 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:32.180184 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:32.180184 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:32.180184 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:32.180184 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:32.180184 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:32.181288 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:32.181288 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:32.181288 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:32.181288 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:32.181288 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:32.181288 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:32.181288 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:32.181288 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:32.181288 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:32.181288 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:32.181288 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 01:43:32.181288 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:43:32.183776 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 143/289] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:43:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:43:32.189540 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e5a6e8-d45c-4938-91d1-98189965bc1a None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:32.369132 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e5a6e8-d45c-4938-91d1-98189965bc1a None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:32.413289 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024e942-2632-431a-8978-a963ea369aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:32.414233 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024e942-2632-431a-8978-a963ea369aed None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:32.414646 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024e942-2632-431a-8978-a963ea369aed None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:32.415050 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024e942-2632-431a-8978-a963ea369aed None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:32.415957 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6024e942-2632-431a-8978-a963ea369aed None admin] RBAC: Adding query filter params (name=member) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:32.416411 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6024e942-2632-431a-8978-a963ea369aed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:32.417930 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6024e942-2632-431a-8978-a963ea369aed None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:32.424641 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:32.432142 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 147/290] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jun 23 01:43:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:32.440307 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54a389-6823-4385-b27c-9c095a525805 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:32.575031 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54a389-6823-4385-b27c-9c095a525805 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:32.628480 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541afa9e-a75b-4a01-9caf-aa97030e3f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:32.629379 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541afa9e-a75b-4a01-9caf-aa97030e3f6f None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:32.629664 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541afa9e-a75b-4a01-9caf-aa97030e3f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:32.629976 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541afa9e-a75b-4a01-9caf-aa97030e3f6f None admin] PATH_INFO: `/v3/users/560712a0e3f54bbfb07a744fe6429fce` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:32.631228 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541afa9e-a75b-4a01-9caf-aa97030e3f6f None admin] RBAC: Authorizing `identity:get_user(user_id=560712a0e3f54bbfb07a744fe6429fce)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:32.632704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541afa9e-a75b-4a01-9caf-aa97030e3f6f None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:32.634355 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 144/291] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:43:32 2022] GET /identity/v3/users/560712a0e3f54bbfb07a744fe6429fce => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:32.640291 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024e942-2632-431a-8978-a963ea369aed None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:32.792021 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024e942-2632-431a-8978-a963ea369aed None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:32.827054 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d180da5-cdb7-4d39-9c75-db0b67f7f202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:32.828042 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d180da5-cdb7-4d39-9c75-db0b67f7f202 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:32.828517 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d180da5-cdb7-4d39-9c75-db0b67f7f202 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:32.828983 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d180da5-cdb7-4d39-9c75-db0b67f7f202 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:32.897742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:32.901762 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d180da5-cdb7-4d39-9c75-db0b67f7f202 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:32.903712 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d180da5-cdb7-4d39-9c75-db0b67f7f202 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:33.003911 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 148/292] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:43:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 01:43:33.009886 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541afa9e-a75b-4a01-9caf-aa97030e3f6f None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:33.116637 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:33.170883 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541afa9e-a75b-4a01-9caf-aa97030e3f6f None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:33.221224 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4a2896-2916-41fe-9a82-c860acbf2689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:33.222071 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a2896-2916-41fe-9a82-c860acbf2689 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:33.222352 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a2896-2916-41fe-9a82-c860acbf2689 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:33.222626 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a2896-2916-41fe-9a82-c860acbf2689 None admin] PATH_INFO: `/v3/projects/9db43d2e143f49efbddf812fd9d03c69` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:33.223945 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4a2896-2916-41fe-9a82-c860acbf2689 None admin] RBAC: Authorizing `identity:get_project(project_id=9db43d2e143f49efbddf812fd9d03c69)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:33.225389 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4a2896-2916-41fe-9a82-c860acbf2689 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:33.227415 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 145/293] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:43:33 2022] GET /identity/v3/projects/9db43d2e143f49efbddf812fd9d03c69 => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:33.233169 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d180da5-cdb7-4d39-9c75-db0b67f7f202 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:33.384749 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:33.389585 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d180da5-cdb7-4d39-9c75-db0b67f7f202 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:33.441745 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a2e3aa-8c7c-46de-ac40-df2ff44d9b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:33.442448 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a2e3aa-8c7c-46de-ac40-df2ff44d9b8b None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:33.442729 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a2e3aa-8c7c-46de-ac40-df2ff44d9b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:33.443003 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a2e3aa-8c7c-46de-ac40-df2ff44d9b8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:33.443888 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a2e3aa-8c7c-46de-ac40-df2ff44d9b8b None admin] RBAC: Adding query filter params (role.id=9021e3c4e3384a309951be4cc1d852e5, scope.project.id=9db43d2e143f49efbddf812fd9d03c69, user.id=560712a0e3f54bbfb07a744fe6429fce) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:33.444202 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a2e3aa-8c7c-46de-ac40-df2ff44d9b8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:33.445646 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a2e3aa-8c7c-46de-ac40-df2ff44d9b8b None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:33.455440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 149/294] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:43:33 2022] GET /identity/v3/role_assignments?user.id=560712a0e3f54bbfb07a744fe6429fce&scope.project.id=9db43d2e143f49efbddf812fd9d03c69&role.id=9021e3c4e3384a309951be4cc1d852e5 => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:33.770010 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf366c45-35f3-44ed-87c2-ae902694072f None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:33.770594 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf366c45-35f3-44ed-87c2-ae902694072f None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:33.770594 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf366c45-35f3-44ed-87c2-ae902694072f None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:33.772120 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 146/295] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:33.777115 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67313676-cf59-43be-a8c7-a70c1abaf3bc None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:33.777522 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67313676-cf59-43be-a8c7-a70c1abaf3bc None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:33.777704 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67313676-cf59-43be-a8c7-a70c1abaf3bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:33.797079 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-67313676-cf59-43be-a8c7-a70c1abaf3bc None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:33.798254 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-67313676-cf59-43be-a8c7-a70c1abaf3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:33.872450 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 150/296] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:33.895240 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf366c45-35f3-44ed-87c2-ae902694072f None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:34.002620 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9c6d7-9d9c-4d31-a0d5-620e9d4ae745 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:34.002967 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9c6d7-9d9c-4d31-a0d5-620e9d4ae745 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:34.003194 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9c6d7-9d9c-4d31-a0d5-620e9d4ae745 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:34.004963 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 151/297] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 01:43:34.010112 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef765389-91cc-4392-ab15-9ada670a25de None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:34.010394 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef765389-91cc-4392-ab15-9ada670a25de None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:34.010665 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef765389-91cc-4392-ab15-9ada670a25de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:34.024444 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf366c45-35f3-44ed-87c2-ae902694072f None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:34.033691 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-ef765389-91cc-4392-ab15-9ada670a25de None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:34.034878 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-ef765389-91cc-4392-ab15-9ada670a25de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:34.073158 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:34.086239 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c68b291-5ee7-4a20-a822-d79439f105fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:34.087944 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68b291-5ee7-4a20-a822-d79439f105fb None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:34.088853 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68b291-5ee7-4a20-a822-d79439f105fb None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:34.089621 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68b291-5ee7-4a20-a822-d79439f105fb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:34.104574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c68b291-5ee7-4a20-a822-d79439f105fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:34.112549 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 152/298] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 01:43:34.115070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-4c68b291-5ee7-4a20-a822-d79439f105fb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 01:43:34.115070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:34.115070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:34.115070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:34.115070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:34.115070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:34.115070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:34.115070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:34.115070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:34.115070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:34.115070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:34.115070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:34.115070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:34.115070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:34.115070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:34.115070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:34.115942 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:34.115942 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:34.115942 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:34.115942 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:34.115942 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:34.115942 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:34.115942 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:34.115942 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:34.115942 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:34.115942 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:34.115942 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:34.115942 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:34.115942 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:34.115942 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:34.115942 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:34.115942 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:34.116738 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:34.116738 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:34.116738 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:34.116738 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:34.116738 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:34.116738 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:34.116738 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:34.116738 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:34.116738 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:34.116738 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:34.116738 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 01:43:34.116738 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:43:34.118658 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 147/299] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:43:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:43:34.124866 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353063e6-875c-47a1-8d0b-5b72e42442b6 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:34.124866 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353063e6-875c-47a1-8d0b-5b72e42442b6 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:34.124866 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353063e6-875c-47a1-8d0b-5b72e42442b6 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:34.124866 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 153/300] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:34.125161 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c68b291-5ee7-4a20-a822-d79439f105fb None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:34.126632 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d038f52-1cf1-43bc-9e19-713a19bbcb58 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:34.126915 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d038f52-1cf1-43bc-9e19-713a19bbcb58 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:34.127195 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d038f52-1cf1-43bc-9e19-713a19bbcb58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:34.145331 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-9d038f52-1cf1-43bc-9e19-713a19bbcb58 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:34.146586 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-9d038f52-1cf1-43bc-9e19-713a19bbcb58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:34.165904 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:34.218308 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 154/301] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 01:43:34.218957 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d038f52-1cf1-43bc-9e19-713a19bbcb58 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:34.234099 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c68b291-5ee7-4a20-a822-d79439f105fb None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:34.271493 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd87408b-4b5c-437c-8a80-3d44c5ef690b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:34.272166 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd87408b-4b5c-437c-8a80-3d44c5ef690b None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:34.272447 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd87408b-4b5c-437c-8a80-3d44c5ef690b None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:34.272724 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd87408b-4b5c-437c-8a80-3d44c5ef690b None admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:34.273484 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd87408b-4b5c-437c-8a80-3d44c5ef690b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:34.273791 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd87408b-4b5c-437c-8a80-3d44c5ef690b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:34.275291 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd87408b-4b5c-437c-8a80-3d44c5ef690b None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:34.283616 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 148/302] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jun 23 01:43:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:34.284828 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef588b-d01b-4682-9e5a-a2f52d7e9fae None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:34.285114 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef588b-d01b-4682-9e5a-a2f52d7e9fae None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:34.285386 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef588b-d01b-4682-9e5a-a2f52d7e9fae None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:34.286795 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 149/303] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:34.290837 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ef588b-d01b-4682-9e5a-a2f52d7e9fae None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:34.338093 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d038f52-1cf1-43bc-9e19-713a19bbcb58 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:34.345187 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:34.378805 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d9590d-7edc-44fa-befa-ea1aeb2b421e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:34.379647 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d9590d-7edc-44fa-befa-ea1aeb2b421e None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:34.379931 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d9590d-7edc-44fa-befa-ea1aeb2b421e None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:34.380214 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d9590d-7edc-44fa-befa-ea1aeb2b421e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:34.381509 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d9590d-7edc-44fa-befa-ea1aeb2b421e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:34.382993 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d9590d-7edc-44fa-befa-ea1aeb2b421e None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:34.384684 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 155/304] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:43:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:34.385449 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d9590d-7edc-44fa-befa-ea1aeb2b421e None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:34.396285 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ef588b-d01b-4682-9e5a-a2f52d7e9fae None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:34.412921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:34.429811 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa8b24b-5367-4196-b8dd-9d4cfab65060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:34.430914 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8b24b-5367-4196-b8dd-9d4cfab65060 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:34.431189 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8b24b-5367-4196-b8dd-9d4cfab65060 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:34.431487 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8b24b-5367-4196-b8dd-9d4cfab65060 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:34.441558 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa8b24b-5367-4196-b8dd-9d4cfab65060 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:34.449214 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-1fa8b24b-5367-4196-b8dd-9d4cfab65060 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 01:43:34.449214 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:34.449214 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:34.449214 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:34.449214 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:34.449214 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:34.449214 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:34.449214 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:34.449214 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:34.449214 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:34.449214 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:34.449214 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:34.449214 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:34.449214 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:34.449214 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:34.449214 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:34.450259 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:34.450259 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:34.450259 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:34.450259 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:34.450259 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:34.450259 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:34.450259 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:34.450259 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:34.450259 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:34.450259 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:34.450259 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:34.450259 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:34.450259 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:34.450259 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:34.450259 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:34.450259 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:34.451052 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:34.451052 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:34.451052 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:34.451052 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:34.451052 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:34.451052 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:34.451052 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:34.451052 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:34.451052 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:34.451052 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:34.451052 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 01:43:34.451052 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:43:34.451711 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 150/305] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:43:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:43:34.452931 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45921e1-69e6-40e6-b308-887ee33005e4 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:34.453406 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45921e1-69e6-40e6-b308-887ee33005e4 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:34.453684 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45921e1-69e6-40e6-b308-887ee33005e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:34.471124 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-d45921e1-69e6-40e6-b308-887ee33005e4 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:34.472373 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-d45921e1-69e6-40e6-b308-887ee33005e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:34.491042 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d9590d-7edc-44fa-befa-ea1aeb2b421e None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:34.525267 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4607f34-91ac-47bd-8dfc-3fa38661d088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:34.526231 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4607f34-91ac-47bd-8dfc-3fa38661d088 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:34.526442 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4607f34-91ac-47bd-8dfc-3fa38661d088 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:34.526710 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4607f34-91ac-47bd-8dfc-3fa38661d088 None admin] PATH_INFO: `/v3/users/10391bebe4494b069bcaa54cadd78f55` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:34.536785 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 151/306] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:34.551430 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45921e1-69e6-40e6-b308-887ee33005e4 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:34.551547 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4607f34-91ac-47bd-8dfc-3fa38661d088 None admin] RBAC: Authorizing `identity:get_user(user_id=10391bebe4494b069bcaa54cadd78f55)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:34.551547 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4607f34-91ac-47bd-8dfc-3fa38661d088 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:34.551547 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 156/307] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:43:34 2022] GET /identity/v3/users/10391bebe4494b069bcaa54cadd78f55 => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:34.551547 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4607f34-91ac-47bd-8dfc-3fa38661d088 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:34.583002 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:34.623062 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:34.638086 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45921e1-69e6-40e6-b308-887ee33005e4 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:34.646167 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4607f34-91ac-47bd-8dfc-3fa38661d088 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:34.671061 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ef329-040f-4f91-b699-2b88f84cfc77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:34.671736 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ef329-040f-4f91-b699-2b88f84cfc77 None admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:34.672018 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ef329-040f-4f91-b699-2b88f84cfc77 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:34.672290 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ef329-040f-4f91-b699-2b88f84cfc77 None admin] PATH_INFO: `/v3/groups` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:34.672980 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ef329-040f-4f91-b699-2b88f84cfc77 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:34.674437 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ef329-040f-4f91-b699-2b88f84cfc77 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:34.679934 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871b06d-7fdf-4c82-ab6d-c6cb3560c8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:34.681030 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871b06d-7fdf-4c82-ab6d-c6cb3560c8fa None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:34.681381 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871b06d-7fdf-4c82-ab6d-c6cb3560c8fa None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:34.681668 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871b06d-7fdf-4c82-ab6d-c6cb3560c8fa None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:34.682613 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4871b06d-7fdf-4c82-ab6d-c6cb3560c8fa None admin] RBAC: Adding query filter params (name=reader) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:34.682905 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4871b06d-7fdf-4c82-ab6d-c6cb3560c8fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:34.684449 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4871b06d-7fdf-4c82-ab6d-c6cb3560c8fa None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:34.689434 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 152/308] 10.222.0.26 () {60 vars in 1134 bytes} [Thu Jun 23 01:43:34 2022] POST /identity/v3/groups => generated 229 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:43:34.690152 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ef329-040f-4f91-b699-2b88f84cfc77 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:34.694970 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 157/309] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jun 23 01:43:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:34.697116 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871b06d-7fdf-4c82-ab6d-c6cb3560c8fa None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:34.818317 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ef329-040f-4f91-b699-2b88f84cfc77 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:34.818838 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871b06d-7fdf-4c82-ab6d-c6cb3560c8fa None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:34.820110 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:34.859750 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d39b1-a37a-4f1c-992a-ce0f0654e3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:34.860514 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d39b1-a37a-4f1c-992a-ce0f0654e3b0 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:34.860795 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d39b1-a37a-4f1c-992a-ce0f0654e3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:34.861420 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d39b1-a37a-4f1c-992a-ce0f0654e3b0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:34.861521 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08e1924-0ebb-43b8-8b96-a1ff7eea2bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:34.861729 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e1924-0ebb-43b8-8b96-a1ff7eea2bdb None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:34.862003 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e1924-0ebb-43b8-8b96-a1ff7eea2bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:34.862277 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e1924-0ebb-43b8-8b96-a1ff7eea2bdb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:34.863121 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08e1924-0ebb-43b8-8b96-a1ff7eea2bdb None admin] RBAC: Adding query filter params (role.id=d159e6edeef94e418a4616504e214063, user.id=10391bebe4494b069bcaa54cadd78f55, scope.system=all) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:34.863550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08e1924-0ebb-43b8-8b96-a1ff7eea2bdb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:34.865139 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08e1924-0ebb-43b8-8b96-a1ff7eea2bdb None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:34.869517 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:34.871709 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 153/310] 10.222.0.26 () {56 vars in 1308 bytes} [Thu Jun 23 01:43:34 2022] GET /identity/v3/role_assignments?user.id=10391bebe4494b069bcaa54cadd78f55&scope.system=all&role.id=d159e6edeef94e418a4616504e214063 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:34.879317 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08e1924-0ebb-43b8-8b96-a1ff7eea2bdb None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:34.879525 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d39b1-a37a-4f1c-992a-ce0f0654e3b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:34.883449 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-fc7d39b1-a37a-4f1c-992a-ce0f0654e3b0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 01:43:34.883449 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:34.883449 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:34.883449 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:34.883449 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:34.883449 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:34.883449 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:34.883449 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:34.883449 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:34.883449 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:34.883449 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:34.883449 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:34.883449 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:34.883449 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:34.883449 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:34.883449 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:34.884572 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:34.884572 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:34.884572 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:34.884572 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:34.884572 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:34.884572 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:34.884572 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:34.884572 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:34.884572 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:34.884572 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:34.884572 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:34.884572 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:34.884572 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:34.884572 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:34.884572 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:34.884572 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:34.885354 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:34.885354 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:34.885354 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:34.885354 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:34.885354 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:34.885354 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:34.885354 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:34.885354 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:34.885354 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:34.885354 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:34.885354 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 01:43:34.885354 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:43:34.887994 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 158/311] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:43:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:43:34.894191 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d39b1-a37a-4f1c-992a-ce0f0654e3b0 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:35.004140 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08e1924-0ebb-43b8-8b96-a1ff7eea2bdb None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:35.044850 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d39b1-a37a-4f1c-992a-ce0f0654e3b0 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:35.045043 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f1c76d-65c8-4336-adc6-450190d71757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:35.045912 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f1c76d-65c8-4336-adc6-450190d71757 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:35.046189 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f1c76d-65c8-4336-adc6-450190d71757 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:35.046455 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f1c76d-65c8-4336-adc6-450190d71757 None admin] PATH_INFO: `/v3/users/af8e2d3b374b48e398c5826f0704f0a3` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:35.047912 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f1c76d-65c8-4336-adc6-450190d71757 None admin] RBAC: Authorizing `identity:get_user(user_id=af8e2d3b374b48e398c5826f0704f0a3)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:35.049233 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f1c76d-65c8-4336-adc6-450190d71757 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:35.051037 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 154/312] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:43:34 2022] GET /identity/v3/users/af8e2d3b374b48e398c5826f0704f0a3 => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:35.056768 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f1c76d-65c8-4336-adc6-450190d71757 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:35.085263 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7324d3a4-8653-4d34-8b96-345be4ca74c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:35.086150 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7324d3a4-8653-4d34-8b96-345be4ca74c8 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:35.086435 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7324d3a4-8653-4d34-8b96-345be4ca74c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:35.086708 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7324d3a4-8653-4d34-8b96-345be4ca74c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:35.087666 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7324d3a4-8653-4d34-8b96-345be4ca74c8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:35.088295 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7324d3a4-8653-4d34-8b96-345be4ca74c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:35.090107 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7324d3a4-8653-4d34-8b96-345be4ca74c8 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:35.097741 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:35.101398 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 159/313] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jun 23 01:43:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:35.106708 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7324d3a4-8653-4d34-8b96-345be4ca74c8 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:35.172109 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f1c76d-65c8-4336-adc6-450190d71757 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:35.197303 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:35.220917 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bd3e47-d0e6-47e3-9b0a-00010447971c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:35.221908 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bd3e47-d0e6-47e3-9b0a-00010447971c None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:35.222364 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bd3e47-d0e6-47e3-9b0a-00010447971c None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:35.223025 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bd3e47-d0e6-47e3-9b0a-00010447971c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:35.246086 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7324d3a4-8653-4d34-8b96-345be4ca74c8 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:35.289283 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdbaf76-6563-409d-8e4e-ed8a2c06b9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:35.290546 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdbaf76-6563-409d-8e4e-ed8a2c06b9c9 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:35.290893 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdbaf76-6563-409d-8e4e-ed8a2c06b9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:35.291260 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdbaf76-6563-409d-8e4e-ed8a2c06b9c9 None admin] PATH_INFO: `/v3/users/f2ea425f1e674bb09fe0041dddccf44d` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:35.293204 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdbaf76-6563-409d-8e4e-ed8a2c06b9c9 None admin] RBAC: Authorizing `identity:get_user(user_id=f2ea425f1e674bb09fe0041dddccf44d)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:35.294861 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdbaf76-6563-409d-8e4e-ed8a2c06b9c9 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:35.297810 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 160/314] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:43:35 2022] GET /identity/v3/users/f2ea425f1e674bb09fe0041dddccf44d => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:35.302799 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdbaf76-6563-409d-8e4e-ed8a2c06b9c9 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:35.322160 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bd3e47-d0e6-47e3-9b0a-00010447971c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:35.324633 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bd3e47-d0e6-47e3-9b0a-00010447971c None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:35.371639 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:35.412224 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdbaf76-6563-409d-8e4e-ed8a2c06b9c9 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:35.443587 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:35.443922 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 155/315] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:43:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 01:43:35.455037 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb78a7ec-1735-45f6-8588-b7cf1e0bf96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:35.455037 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb78a7ec-1735-45f6-8588-b7cf1e0bf96e None admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:35.455037 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb78a7ec-1735-45f6-8588-b7cf1e0bf96e None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:35.455037 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb78a7ec-1735-45f6-8588-b7cf1e0bf96e None admin] PATH_INFO: `/v3/system/users/f2ea425f1e674bb09fe0041dddccf44d/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:35.455435 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bd3e47-d0e6-47e3-9b0a-00010447971c None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:35.455505 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb78a7ec-1735-45f6-8588-b7cf1e0bf96e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f2ea425f1e674bb09fe0041dddccf44d, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:35.455505 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb78a7ec-1735-45f6-8588-b7cf1e0bf96e None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:35.466116 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 161/316] 10.222.0.26 () {58 vars in 1334 bytes} [Thu Jun 23 01:43:35 2022] PUT /identity/v3/system/users/f2ea425f1e674bb09fe0041dddccf44d/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 01:43:35.563365 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bd3e47-d0e6-47e3-9b0a-00010447971c None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:35.600313 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720f1347-0bb5-4f60-b26b-335b336b2220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:35.601186 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f1347-0bb5-4f60-b26b-335b336b2220 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:35.601465 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f1347-0bb5-4f60-b26b-335b336b2220 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:35.601735 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f1347-0bb5-4f60-b26b-335b336b2220 None admin] PATH_INFO: `/v3/projects/9db43d2e143f49efbddf812fd9d03c69` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:35.602975 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720f1347-0bb5-4f60-b26b-335b336b2220 None admin] RBAC: Authorizing `identity:get_project(project_id=9db43d2e143f49efbddf812fd9d03c69)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:35.604483 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720f1347-0bb5-4f60-b26b-335b336b2220 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:35.611173 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 156/317] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:43:35 2022] GET /identity/v3/projects/9db43d2e143f49efbddf812fd9d03c69 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:35.612638 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb78a7ec-1735-45f6-8588-b7cf1e0bf96e None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:35.717538 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb78a7ec-1735-45f6-8588-b7cf1e0bf96e None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:35.751252 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc88a01a-b98f-4ee1-8496-8e325b2c7e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:35.751915 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88a01a-b98f-4ee1-8496-8e325b2c7e92 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:35.752193 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88a01a-b98f-4ee1-8496-8e325b2c7e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:35.752465 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88a01a-b98f-4ee1-8496-8e325b2c7e92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:35.753294 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc88a01a-b98f-4ee1-8496-8e325b2c7e92 None admin] RBAC: Adding query filter params (role.id=d159e6edeef94e418a4616504e214063, scope.project.id=9db43d2e143f49efbddf812fd9d03c69, user.id=af8e2d3b374b48e398c5826f0704f0a3) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:35.753606 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc88a01a-b98f-4ee1-8496-8e325b2c7e92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:35.755074 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc88a01a-b98f-4ee1-8496-8e325b2c7e92 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:35.761092 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 162/318] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:43:35 2022] GET /identity/v3/role_assignments?user.id=af8e2d3b374b48e398c5826f0704f0a3&scope.project.id=9db43d2e143f49efbddf812fd9d03c69&role.id=d159e6edeef94e418a4616504e214063 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:36.743806 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67de80-1c9f-4cda-9a41-a14926af01b4 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:36.744413 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67de80-1c9f-4cda-9a41-a14926af01b4 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:36.744413 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67de80-1c9f-4cda-9a41-a14926af01b4 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:36.747254 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 157/319] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:36.750921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd81a60-a5fd-4dff-bdc6-5c08f7d07ff6 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:36.751720 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd81a60-a5fd-4dff-bdc6-5c08f7d07ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:36.752220 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd81a60-a5fd-4dff-bdc6-5c08f7d07ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:36.769808 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-0bd81a60-a5fd-4dff-bdc6-5c08f7d07ff6 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:36.771307 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-0bd81a60-a5fd-4dff-bdc6-5c08f7d07ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:36.826210 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:36.842055 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 163/320] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:36.864216 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a67de80-1c9f-4cda-9a41-a14926af01b4 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:36.867309 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:36.948507 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903dfd7-95b1-4a08-8970-71169b907c2b None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:36.949072 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903dfd7-95b1-4a08-8970-71169b907c2b None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:36.949354 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903dfd7-95b1-4a08-8970-71169b907c2b None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:36.951131 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 164/321] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:36.956177 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43e151-2879-4890-87fd-af7889031730 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:36.956423 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43e151-2879-4890-87fd-af7889031730 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:36.956733 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43e151-2879-4890-87fd-af7889031730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:36.973340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-4c43e151-2879-4890-87fd-af7889031730 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:36.974586 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-4c43e151-2879-4890-87fd-af7889031730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:36.976769 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a67de80-1c9f-4cda-9a41-a14926af01b4 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:37.017398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e7ea9-313f-4a06-a0cc-e78996292a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:37.018109 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e7ea9-313f-4a06-a0cc-e78996292a87 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:37.018393 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e7ea9-313f-4a06-a0cc-e78996292a87 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:37.018673 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e7ea9-313f-4a06-a0cc-e78996292a87 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:37.019834 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747e7ea9-313f-4a06-a0cc-e78996292a87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:37.021320 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747e7ea9-313f-4a06-a0cc-e78996292a87 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:37.023095 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 158/322] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:43:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:37.028212 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e7ea9-313f-4a06-a0cc-e78996292a87 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:37.043580 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 165/323] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 01:43:37.065794 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c43e151-2879-4890-87fd-af7889031730 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:37.102506 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:37.133779 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e7ea9-313f-4a06-a0cc-e78996292a87 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:37.168868 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd1ec8-7c4b-4d79-9cb0-37021bd8a2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:37.169534 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd1ec8-7c4b-4d79-9cb0-37021bd8a2af None admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:37.169814 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd1ec8-7c4b-4d79-9cb0-37021bd8a2af None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:37.170088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd1ec8-7c4b-4d79-9cb0-37021bd8a2af None admin] PATH_INFO: `/v3/groups` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:37.170783 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fd1ec8-7c4b-4d79-9cb0-37021bd8a2af None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:37.172314 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fd1ec8-7c4b-4d79-9cb0-37021bd8a2af None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:37.173306 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c43e151-2879-4890-87fd-af7889031730 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:37.189161 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 159/324] 10.222.0.26 () {60 vars in 1134 bytes} [Thu Jun 23 01:43:37 2022] POST /identity/v3/groups => generated 226 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 01:43:37.192959 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:37.212715 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3600c8ec-7d54-4c3e-bf58-c3513c44741b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:37.213545 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3600c8ec-7d54-4c3e-bf58-c3513c44741b None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:37.213828 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3600c8ec-7d54-4c3e-bf58-c3513c44741b None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:37.214104 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3600c8ec-7d54-4c3e-bf58-c3513c44741b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:37.226495 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3600c8ec-7d54-4c3e-bf58-c3513c44741b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:37.234540 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-3600c8ec-7d54-4c3e-bf58-c3513c44741b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 01:43:37.234540 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:37.234540 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:37.234540 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:37.234540 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:37.234540 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:37.234540 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:37.234540 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:37.234540 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:37.234540 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:37.234540 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:37.234540 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:37.234540 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:37.234540 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:37.234540 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:37.234540 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:37.235860 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:37.235860 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:37.235860 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:37.235860 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:37.235860 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:37.235860 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:37.235860 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:37.235860 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:37.235860 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:37.235860 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:37.235860 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:37.235860 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:37.235860 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:37.235860 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:37.235860 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:37.235860 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:37.236662 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:37.236662 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:37.236662 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:37.236662 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:37.236662 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:37.236662 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:37.236662 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:37.236662 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:37.236662 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:37.236662 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:37.236662 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 01:43:37.236662 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:43:37.238371 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 166/325] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:43:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:43:37.242456 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd1ec8-7c4b-4d79-9cb0-37021bd8a2af None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:37.361042 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd1ec8-7c4b-4d79-9cb0-37021bd8a2af None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:37.390658 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:37.398004 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaba08c-2d8a-4e5c-9da1-f7dfff12bff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:37.398660 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaba08c-2d8a-4e5c-9da1-f7dfff12bff8 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:37.398979 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaba08c-2d8a-4e5c-9da1-f7dfff12bff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:37.399285 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaba08c-2d8a-4e5c-9da1-f7dfff12bff8 None admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:37.400062 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaba08c-2d8a-4e5c-9da1-f7dfff12bff8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:37.400371 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaba08c-2d8a-4e5c-9da1-f7dfff12bff8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:37.402314 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaba08c-2d8a-4e5c-9da1-f7dfff12bff8 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:37.411348 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 160/326] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jun 23 01:43:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:37.417034 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3600c8ec-7d54-4c3e-bf58-c3513c44741b None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:37.526255 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3600c8ec-7d54-4c3e-bf58-c3513c44741b None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:37.534066 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5777b13a-9db0-490f-8382-c1473c1de89e None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:37.534357 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5777b13a-9db0-490f-8382-c1473c1de89e None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:37.534623 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5777b13a-9db0-490f-8382-c1473c1de89e None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:37.537425 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 161/327] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:37.541619 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72672b12-641a-4939-a1fc-4b655adddb26 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:37.541898 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72672b12-641a-4939-a1fc-4b655adddb26 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:37.542171 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72672b12-641a-4939-a1fc-4b655adddb26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:37.561232 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-72672b12-641a-4939-a1fc-4b655adddb26 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:37.562345 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-72672b12-641a-4939-a1fc-4b655adddb26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:37.568881 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ade42cb-bb15-4d38-baf2-5eaa1f73d622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:37.569774 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade42cb-bb15-4d38-baf2-5eaa1f73d622 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:37.570056 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade42cb-bb15-4d38-baf2-5eaa1f73d622 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:37.570333 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade42cb-bb15-4d38-baf2-5eaa1f73d622 None admin] PATH_INFO: `/v3/users/10391bebe4494b069bcaa54cadd78f55` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:37.571824 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ade42cb-bb15-4d38-baf2-5eaa1f73d622 None admin] RBAC: Authorizing `identity:get_user(user_id=10391bebe4494b069bcaa54cadd78f55)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:37.573288 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ade42cb-bb15-4d38-baf2-5eaa1f73d622 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:37.575098 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 167/328] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:43:37 2022] GET /identity/v3/users/10391bebe4494b069bcaa54cadd78f55 => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:37.580526 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ade42cb-bb15-4d38-baf2-5eaa1f73d622 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:37.643805 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:37.651352 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 162/329] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 01:43:37.676416 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72672b12-641a-4939-a1fc-4b655adddb26 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:37.712629 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ade42cb-bb15-4d38-baf2-5eaa1f73d622 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:37.763973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e97f5-dba1-4f28-ab09-91d732bede1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:37.764873 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e97f5-dba1-4f28-ab09-91d732bede1e None admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:37.765157 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e97f5-dba1-4f28-ab09-91d732bede1e None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:37.765441 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e97f5-dba1-4f28-ab09-91d732bede1e None admin] PATH_INFO: `/v3/system/users/10391bebe4494b069bcaa54cadd78f55/roles/d159e6edeef94e418a4616504e214063` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:37.767057 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9e97f5-dba1-4f28-ab09-91d732bede1e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=10391bebe4494b069bcaa54cadd78f55, role_id=d159e6edeef94e418a4616504e214063)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:37.769129 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9e97f5-dba1-4f28-ab09-91d732bede1e None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:37.779879 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 168/330] 10.222.0.26 () {58 vars in 1334 bytes} [Thu Jun 23 01:43:37 2022] PUT /identity/v3/system/users/10391bebe4494b069bcaa54cadd78f55/roles/d159e6edeef94e418a4616504e214063 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 01:43:37.802071 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72672b12-641a-4939-a1fc-4b655adddb26 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:37.831710 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:37.850195 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e166f-2951-42a5-9baf-0a2daf8695ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:37.851380 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e166f-2951-42a5-9baf-0a2daf8695ac None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:37.851798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e166f-2951-42a5-9baf-0a2daf8695ac None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:37.852234 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e166f-2951-42a5-9baf-0a2daf8695ac None admin] PATH_INFO: `/v3/roles/member` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:37.868482 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995e166f-2951-42a5-9baf-0a2daf8695ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:37.876987 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-995e166f-2951-42a5-9baf-0a2daf8695ac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 01:43:37.876987 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:37.876987 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:37.876987 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:37.876987 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:37.876987 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:37.876987 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:37.876987 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:37.876987 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:37.876987 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:37.876987 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:37.876987 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:37.876987 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:37.876987 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:37.876987 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:37.876987 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:37.878146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:37.878146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:37.878146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:37.878146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:37.878146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:37.878146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:37.878146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:37.878146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:37.878146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:37.878146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:37.878146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:37.878146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:37.878146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:37.878146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:37.878146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:37.878146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:37.879117 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:37.879117 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:37.879117 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:37.879117 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:37.879117 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:37.879117 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:37.879117 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:37.879117 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:37.879117 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:37.879117 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:37.879117 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 01:43:37.879117 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:43:37.881704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 163/331] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:43:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:43:37.887591 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e97f5-dba1-4f28-ab09-91d732bede1e None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:38.020462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e97f5-dba1-4f28-ab09-91d732bede1e None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:38.032399 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:38.061941 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505ddf4b-a9bd-488f-9890-8ec59caece08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:38.062858 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ddf4b-a9bd-488f-9890-8ec59caece08 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:38.063347 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ddf4b-a9bd-488f-9890-8ec59caece08 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:38.063781 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ddf4b-a9bd-488f-9890-8ec59caece08 None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:38.064681 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505ddf4b-a9bd-488f-9890-8ec59caece08 None admin] RBAC: Adding query filter params (name=member) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:38.065157 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505ddf4b-a9bd-488f-9890-8ec59caece08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:38.066723 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505ddf4b-a9bd-488f-9890-8ec59caece08 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:38.076326 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 169/332] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jun 23 01:43:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:38.084461 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e166f-2951-42a5-9baf-0a2daf8695ac None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:38.195755 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e166f-2951-42a5-9baf-0a2daf8695ac None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:38.234543 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a209fe-4d8b-4c53-b5b3-5eb3168bc981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:38.235688 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a209fe-4d8b-4c53-b5b3-5eb3168bc981 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:38.236104 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a209fe-4d8b-4c53-b5b3-5eb3168bc981 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:38.236536 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a209fe-4d8b-4c53-b5b3-5eb3168bc981 None admin] PATH_INFO: `/v3/users/f2ea425f1e674bb09fe0041dddccf44d` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:38.237892 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a209fe-4d8b-4c53-b5b3-5eb3168bc981 None admin] RBAC: Authorizing `identity:get_user(user_id=f2ea425f1e674bb09fe0041dddccf44d)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:38.239629 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a209fe-4d8b-4c53-b5b3-5eb3168bc981 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:38.241421 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 164/333] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:43:38 2022] GET /identity/v3/users/f2ea425f1e674bb09fe0041dddccf44d => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:38.247257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505ddf4b-a9bd-488f-9890-8ec59caece08 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:38.355979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505ddf4b-a9bd-488f-9890-8ec59caece08 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:38.390947 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f04132-7488-45a0-83a8-90d98c235295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:38.391868 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f04132-7488-45a0-83a8-90d98c235295 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:38.392283 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f04132-7488-45a0-83a8-90d98c235295 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:38.392716 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f04132-7488-45a0-83a8-90d98c235295 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:38.393692 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f04132-7488-45a0-83a8-90d98c235295 None admin] RBAC: Adding query filter params (role.id=9021e3c4e3384a309951be4cc1d852e5, user.id=f2ea425f1e674bb09fe0041dddccf44d, scope.system=all) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:38.394136 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f04132-7488-45a0-83a8-90d98c235295 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:38.395781 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f04132-7488-45a0-83a8-90d98c235295 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:38.408851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 170/334] 10.222.0.26 () {56 vars in 1308 bytes} [Thu Jun 23 01:43:38 2022] GET /identity/v3/role_assignments?user.id=f2ea425f1e674bb09fe0041dddccf44d&scope.system=all&role.id=9021e3c4e3384a309951be4cc1d852e5 => generated 510 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:43:39.768499 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d047610-1e4c-49ba-bf39-99580e9ca31c None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:39.769009 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d047610-1e4c-49ba-bf39-99580e9ca31c None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:39.769009 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d047610-1e4c-49ba-bf39-99580e9ca31c None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:39.772630 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 165/335] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:39.779540 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac2b48-4db1-4ba1-9af3-6caa3b347fb2 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:39.779970 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac2b48-4db1-4ba1-9af3-6caa3b347fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:39.780140 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac2b48-4db1-4ba1-9af3-6caa3b347fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:39.798149 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-79ac2b48-4db1-4ba1-9af3-6caa3b347fb2 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:39.799434 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-79ac2b48-4db1-4ba1-9af3-6caa3b347fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:39.831381 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:39.879576 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 171/336] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 01:43:39.921595 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d047610-1e4c-49ba-bf39-99580e9ca31c None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:39.985666 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:40.052518 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d047610-1e4c-49ba-bf39-99580e9ca31c None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:40.097335 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b94030-e152-4690-a467-3a162ace185c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:40.098134 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b94030-e152-4690-a467-3a162ace185c None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:40.098501 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b94030-e152-4690-a467-3a162ace185c None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:40.098776 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b94030-e152-4690-a467-3a162ace185c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:40.112366 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b94030-e152-4690-a467-3a162ace185c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:40.120146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-a5b94030-e152-4690-a467-3a162ace185c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 01:43:40.120146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:40.120146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:40.120146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:40.120146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:40.120146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:40.120146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:40.120146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:40.120146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:40.120146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:40.120146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:40.120146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:40.120146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:40.120146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:40.120146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:40.120146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:40.121034 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:40.121034 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:40.121034 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:40.121034 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:40.121034 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:40.121034 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:40.121034 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:40.121034 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:40.121034 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:40.121034 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:40.121034 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:40.121034 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:40.121034 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:40.121034 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:40.121034 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:40.121034 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:40.121898 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:40.121898 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:40.121898 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:40.121898 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:40.121898 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:40.121898 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:40.121898 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:40.121898 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:40.121898 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:40.121898 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:40.121898 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 01:43:40.121898 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:43:40.126147 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 166/337] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:43:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:43:40.127651 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ac2b48-4db1-4ba1-9af3-6caa3b347fb2 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:40.162254 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327ab27-3645-49e8-82fd-c09acfed0fd8 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:40.162557 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327ab27-3645-49e8-82fd-c09acfed0fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:40.162807 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327ab27-3645-49e8-82fd-c09acfed0fd8 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:40.164323 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 167/338] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 01:43:40.169701 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aebb0ca-4803-4087-8c94-4787a24b0e12 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:40.170102 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aebb0ca-4803-4087-8c94-4787a24b0e12 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:40.170388 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aebb0ca-4803-4087-8c94-4787a24b0e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:40.194073 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-5aebb0ca-4803-4087-8c94-4787a24b0e12 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:40.195305 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-5aebb0ca-4803-4087-8c94-4787a24b0e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:40.281536 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ac2b48-4db1-4ba1-9af3-6caa3b347fb2 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:40.295295 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 168/339] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:40.339632 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89843a82-6566-4ffb-a350-b89450b90638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:40.340239 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89843a82-6566-4ffb-a350-b89450b90638 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:40.340517 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89843a82-6566-4ffb-a350-b89450b90638 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:40.340797 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89843a82-6566-4ffb-a350-b89450b90638 None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:40.341350 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aebb0ca-4803-4087-8c94-4787a24b0e12 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:40.341536 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89843a82-6566-4ffb-a350-b89450b90638 None admin] RBAC: Adding query filter params (name=member) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:40.341842 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89843a82-6566-4ffb-a350-b89450b90638 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:40.343235 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89843a82-6566-4ffb-a350-b89450b90638 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:40.356557 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 172/340] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jun 23 01:43:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:40.361824 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89843a82-6566-4ffb-a350-b89450b90638 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:40.397230 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:40.405846 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:40.490447 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aebb0ca-4803-4087-8c94-4787a24b0e12 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:40.501297 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89843a82-6566-4ffb-a350-b89450b90638 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:40.528487 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1693b9f-b533-4711-875b-f08a01cc1d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:40.529638 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1693b9f-b533-4711-875b-f08a01cc1d77 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:40.530161 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1693b9f-b533-4711-875b-f08a01cc1d77 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:40.530607 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1693b9f-b533-4711-875b-f08a01cc1d77 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:40.544761 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1693b9f-b533-4711-875b-f08a01cc1d77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:40.553704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-c1693b9f-b533-4711-875b-f08a01cc1d77 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 01:43:40.553704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:40.553704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:40.553704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:40.553704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:40.553704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:40.553704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:40.553704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:40.553704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:40.553704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:40.553704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:40.553704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:40.553704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:40.553704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:40.553704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:40.553704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:40.554605 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:40.554605 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:40.554605 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:40.554605 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:40.554605 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:40.554605 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:40.554605 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:40.554605 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:40.554605 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:40.554605 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:40.554605 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:40.554605 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:40.554605 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:40.554605 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:40.554605 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:40.554605 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:40.555794 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:40.555794 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:40.555794 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:40.555794 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:40.555794 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:40.555794 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:40.555794 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:40.555794 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:40.555794 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:40.555794 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:40.555794 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 01:43:40.555794 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:43:40.558330 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 169/341] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Jun 23 01:43:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:43:40.563698 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1693b9f-b533-4711-875b-f08a01cc1d77 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:40.563921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffa6337-95cc-4c05-baec-e8f78ed89e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:40.564595 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa6337-95cc-4c05-baec-e8f78ed89e27 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:40.564878 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa6337-95cc-4c05-baec-e8f78ed89e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:40.565297 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa6337-95cc-4c05-baec-e8f78ed89e27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:40.675703 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1693b9f-b533-4711-875b-f08a01cc1d77 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:40.695657 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:40.731386 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e546c0-f6e3-4b7e-bc6a-857b658ee7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:40.732299 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e546c0-f6e3-4b7e-bc6a-857b658ee7b6 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:40.732761 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e546c0-f6e3-4b7e-bc6a-857b658ee7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:40.733177 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e546c0-f6e3-4b7e-bc6a-857b658ee7b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:40.734161 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e546c0-f6e3-4b7e-bc6a-857b658ee7b6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:40.734622 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e546c0-f6e3-4b7e-bc6a-857b658ee7b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:40.736187 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e546c0-f6e3-4b7e-bc6a-857b658ee7b6 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:40.736298 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffa6337-95cc-4c05-baec-e8f78ed89e27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:40.737696 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffa6337-95cc-4c05-baec-e8f78ed89e27 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:40.749802 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 170/342] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:43:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:40.759329 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e546c0-f6e3-4b7e-bc6a-857b658ee7b6 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:40.791575 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:40.884801 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e546c0-f6e3-4b7e-bc6a-857b658ee7b6 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:40.923729 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf618ab-9c5c-4642-8682-0a5b3db3b9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:40.924731 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf618ab-9c5c-4642-8682-0a5b3db3b9e8 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:40.925159 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf618ab-9c5c-4642-8682-0a5b3db3b9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:40.925572 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf618ab-9c5c-4642-8682-0a5b3db3b9e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:40.934518 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 173/343] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:43:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 01:43:40.940444 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd5f75-3e67-4b66-a484-cabd1a5cde38 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:40.940828 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd5f75-3e67-4b66-a484-cabd1a5cde38 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:40.941202 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd5f75-3e67-4b66-a484-cabd1a5cde38 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:40.943784 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 174/344] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:40.945137 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9a3b2-a463-4b82-adb4-0dd765197c62 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:40.945513 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9a3b2-a463-4b82-adb4-0dd765197c62 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:40.945882 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9a3b2-a463-4b82-adb4-0dd765197c62 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:40.947476 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 175/345] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:40.952204 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e9a3b2-a463-4b82-adb4-0dd765197c62 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:40.961953 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:41.008911 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf618ab-9c5c-4642-8682-0a5b3db3b9e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:41.010755 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf618ab-9c5c-4642-8682-0a5b3db3b9e8 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:41.069560 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e9a3b2-a463-4b82-adb4-0dd765197c62 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:41.094195 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:41.108251 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928287ec-db2a-4f34-83e6-d77875e97442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:41.108730 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928287ec-db2a-4f34-83e6-d77875e97442 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:41.109102 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928287ec-db2a-4f34-83e6-d77875e97442 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:41.109488 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928287ec-db2a-4f34-83e6-d77875e97442 None admin] PATH_INFO: `/v3/projects/e67bbf3bcff7420690a69b58a6241244` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:41.110868 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928287ec-db2a-4f34-83e6-d77875e97442 None admin] RBAC: Authorizing `identity:get_project(project_id=e67bbf3bcff7420690a69b58a6241244)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:41.112511 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928287ec-db2a-4f34-83e6-d77875e97442 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:41.119095 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 176/346] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:43:40 2022] GET /identity/v3/projects/e67bbf3bcff7420690a69b58a6241244 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:41.120463 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f178e63-7714-4e1a-9e2e-1ebab7fb2f3c None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:41.120842 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f178e63-7714-4e1a-9e2e-1ebab7fb2f3c None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:41.121215 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f178e63-7714-4e1a-9e2e-1ebab7fb2f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:41.121804 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 171/347] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:43:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 01:43:41.124544 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e21e3f-5da2-4f35-bd87-431a35a92980 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:41.124984 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e21e3f-5da2-4f35-bd87-431a35a92980 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:41.125448 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e21e3f-5da2-4f35-bd87-431a35a92980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:41.139574 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-7f178e63-7714-4e1a-9e2e-1ebab7fb2f3c None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:41.140890 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-7f178e63-7714-4e1a-9e2e-1ebab7fb2f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:41.144429 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-a0e21e3f-5da2-4f35-bd87-431a35a92980 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:41.145889 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-a0e21e3f-5da2-4f35-bd87-431a35a92980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:41.208265 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:41.213758 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 177/348] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:41.214521 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f178e63-7714-4e1a-9e2e-1ebab7fb2f3c None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:41.221648 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 172/349] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:41.222679 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e21e3f-5da2-4f35-bd87-431a35a92980 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:41.330260 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e21e3f-5da2-4f35-bd87-431a35a92980 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:41.354033 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f178e63-7714-4e1a-9e2e-1ebab7fb2f3c None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:41.371031 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e043e422-4303-4559-a6d9-717b4524aea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:41.372302 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043e422-4303-4559-a6d9-717b4524aea2 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:41.372783 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043e422-4303-4559-a6d9-717b4524aea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:41.373327 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043e422-4303-4559-a6d9-717b4524aea2 None admin] PATH_INFO: `/v3/groups/7a6d9470aae44aa9aa3f20c238c5a421` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:41.376694 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:41.382222 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e043e422-4303-4559-a6d9-717b4524aea2 None admin] RBAC: Authorizing `identity:get_group(group_id=7a6d9470aae44aa9aa3f20c238c5a421)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:41.384102 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e043e422-4303-4559-a6d9-717b4524aea2 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:41.386246 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 173/350] 10.222.0.26 () {56 vars in 1180 bytes} [Thu Jun 23 01:43:41 2022] GET /identity/v3/groups/7a6d9470aae44aa9aa3f20c238c5a421 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:41.387233 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e043e422-4303-4559-a6d9-717b4524aea2 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:41.398002 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00694898-23c0-47a0-a048-499e0dad08a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:41.398873 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00694898-23c0-47a0-a048-499e0dad08a9 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:41.399147 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00694898-23c0-47a0-a048-499e0dad08a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:41.399687 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00694898-23c0-47a0-a048-499e0dad08a9 None admin] PATH_INFO: `/v3/projects/e67bbf3bcff7420690a69b58a6241244` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:41.400900 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00694898-23c0-47a0-a048-499e0dad08a9 None admin] RBAC: Authorizing `identity:get_project(project_id=e67bbf3bcff7420690a69b58a6241244)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:41.402502 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00694898-23c0-47a0-a048-499e0dad08a9 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:41.404259 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 178/351] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:43:41 2022] GET /identity/v3/projects/e67bbf3bcff7420690a69b58a6241244 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:41.405190 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00694898-23c0-47a0-a048-499e0dad08a9 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:41.456706 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:41.505497 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e043e422-4303-4559-a6d9-717b4524aea2 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:41.511989 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00694898-23c0-47a0-a048-499e0dad08a9 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:41.542473 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f61259-c4fd-4c5f-9e47-8d766b2694a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:41.543788 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f61259-c4fd-4c5f-9e47-8d766b2694a7 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:41.544294 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f61259-c4fd-4c5f-9e47-8d766b2694a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:41.544788 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f61259-c4fd-4c5f-9e47-8d766b2694a7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:41.548383 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d3cf4-24f5-4b9a-8658-6e218263a83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:41.549190 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d3cf4-24f5-4b9a-8658-6e218263a83e None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:41.549466 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d3cf4-24f5-4b9a-8658-6e218263a83e None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:41.549736 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d3cf4-24f5-4b9a-8658-6e218263a83e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:41.558439 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f61259-c4fd-4c5f-9e47-8d766b2694a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:41.561119 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8d3cf4-24f5-4b9a-8658-6e218263a83e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:41.567377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-55f61259-c4fd-4c5f-9e47-8d766b2694a7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 01:43:41.567377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:41.567377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:41.567377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:41.567377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:41.567377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:41.567377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:41.567377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:41.567377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:41.567377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:41.567377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:41.567377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:41.567377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:41.567377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:41.567377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:41.567377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:41.568323 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:41.568323 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:41.568323 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:41.568323 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:41.568323 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:41.568323 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:41.568323 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:41.568323 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:41.568323 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:41.568323 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:41.568323 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:41.568323 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:41.568323 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:41.568323 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:41.568323 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:41.568323 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:41.569144 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:41.569144 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:41.569144 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:41.569144 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:41.569144 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:41.569144 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:41.569144 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:41.569144 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:41.569144 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:41.569144 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:41.569144 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 01:43:41.569144 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:43:41.569752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-4b8d3cf4-24f5-4b9a-8658-6e218263a83e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 01:43:41.569752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:41.569752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:41.569752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:41.569752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:41.569752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:41.569752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:41.569752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:41.569752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:41.569752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:41.569752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:41.569752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:41.569752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:41.569752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:41.569752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:41.569752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:41.570541 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:41.570541 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:41.570541 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:41.570541 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:41.570541 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:41.570541 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:41.570541 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:41.570541 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:41.570541 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:41.570541 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:41.570541 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:41.570541 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:41.570541 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:41.570541 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:41.570541 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:41.570541 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:41.571336 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:41.571336 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:41.571336 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:41.571336 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:41.571336 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:41.571336 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:41.571336 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:41.571336 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:41.571336 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:41.571336 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:41.571336 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 01:43:41.571336 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:43:41.574143 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 174/353] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:43:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:43:41.574363 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 179/353] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:43:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:43:41.575265 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d3cf4-24f5-4b9a-8658-6e218263a83e None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:41.575586 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f61259-c4fd-4c5f-9e47-8d766b2694a7 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:41.631304 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:41.687604 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f61259-c4fd-4c5f-9e47-8d766b2694a7 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:41.703870 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d3cf4-24f5-4b9a-8658-6e218263a83e None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:41.708432 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:41.732484 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015d6da1-6393-407e-9ed7-d7e791aae7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:41.732484 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015d6da1-6393-407e-9ed7-d7e791aae7ed None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:41.732484 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015d6da1-6393-407e-9ed7-d7e791aae7ed None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:41.732484 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015d6da1-6393-407e-9ed7-d7e791aae7ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:41.732871 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015d6da1-6393-407e-9ed7-d7e791aae7ed None admin] RBAC: Adding query filter params (group.id=7a6d9470aae44aa9aa3f20c238c5a421, role.id=9021e3c4e3384a309951be4cc1d852e5, scope.project.id=e67bbf3bcff7420690a69b58a6241244) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:41.732871 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015d6da1-6393-407e-9ed7-d7e791aae7ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:41.734605 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015d6da1-6393-407e-9ed7-d7e791aae7ed None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:41.742136 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 175/354] 10.222.0.26 () {56 vars in 1376 bytes} [Thu Jun 23 01:43:41 2022] GET /identity/v3/role_assignments?group.id=7a6d9470aae44aa9aa3f20c238c5a421&scope.project.id=e67bbf3bcff7420690a69b58a6241244&role.id=9021e3c4e3384a309951be4cc1d852e5 => generated 260 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:41.743922 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015d6da1-6393-407e-9ed7-d7e791aae7ed None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:41.746934 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb2299a-5dee-43ac-99eb-8909ced1a3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:41.747909 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb2299a-5dee-43ac-99eb-8909ced1a3b1 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:41.749136 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb2299a-5dee-43ac-99eb-8909ced1a3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:41.749136 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb2299a-5dee-43ac-99eb-8909ced1a3b1 None admin] PATH_INFO: `/v3/groups/7a6d9470aae44aa9aa3f20c238c5a421` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:41.750044 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb2299a-5dee-43ac-99eb-8909ced1a3b1 None admin] RBAC: Authorizing `identity:get_group(group_id=7a6d9470aae44aa9aa3f20c238c5a421)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:41.751639 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb2299a-5dee-43ac-99eb-8909ced1a3b1 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:41.753647 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 180/355] 10.222.0.26 () {56 vars in 1180 bytes} [Thu Jun 23 01:43:41 2022] GET /identity/v3/groups/7a6d9470aae44aa9aa3f20c238c5a421 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:41.754339 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb2299a-5dee-43ac-99eb-8909ced1a3b1 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:41.861589 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015d6da1-6393-407e-9ed7-d7e791aae7ed None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:41.873309 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb2299a-5dee-43ac-99eb-8909ced1a3b1 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:41.892433 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:41.908494 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0965e4db-61b5-4338-a4f5-cb6f4d1e0290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:41.909181 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965e4db-61b5-4338-a4f5-cb6f4d1e0290 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:41.909449 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965e4db-61b5-4338-a4f5-cb6f4d1e0290 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:41.909717 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965e4db-61b5-4338-a4f5-cb6f4d1e0290 None admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:41.910490 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0965e4db-61b5-4338-a4f5-cb6f4d1e0290 None admin] RBAC: Adding query filter params (name=member) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:41.910833 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0965e4db-61b5-4338-a4f5-cb6f4d1e0290 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:41.912362 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0965e4db-61b5-4338-a4f5-cb6f4d1e0290 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:41.923472 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe27c633-4bfd-4b82-b130-7b239d8e135b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:41.924262 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 176/356] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jun 23 01:43:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:41.924527 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe27c633-4bfd-4b82-b130-7b239d8e135b None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:41.926165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe27c633-4bfd-4b82-b130-7b239d8e135b None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:41.926165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe27c633-4bfd-4b82-b130-7b239d8e135b None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:41.926325 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b609150-2b70-4c7b-81b6-e11d61571d76 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:41.926325 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b609150-2b70-4c7b-81b6-e11d61571d76 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:41.926545 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe27c633-4bfd-4b82-b130-7b239d8e135b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:41.926614 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b609150-2b70-4c7b-81b6-e11d61571d76 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:41.926679 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe27c633-4bfd-4b82-b130-7b239d8e135b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:41.927697 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 177/357] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:41.927793 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe27c633-4bfd-4b82-b130-7b239d8e135b None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:41.928925 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b609150-2b70-4c7b-81b6-e11d61571d76 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:41.938165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 181/358] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jun 23 01:43:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:41.943071 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe27c633-4bfd-4b82-b130-7b239d8e135b None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:41.997615 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:42.060961 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe27c633-4bfd-4b82-b130-7b239d8e135b None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:42.079743 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b609150-2b70-4c7b-81b6-e11d61571d76 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:42.100392 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ec5fd-835c-45ab-b8dc-582a912d0931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:42.101057 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ec5fd-835c-45ab-b8dc-582a912d0931 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:42.101396 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ec5fd-835c-45ab-b8dc-582a912d0931 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:42.101681 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ec5fd-835c-45ab-b8dc-582a912d0931 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:42.123079 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5f0484-05b6-4dbf-b8ec-317c6c40b42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:42.124034 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f0484-05b6-4dbf-b8ec-317c6c40b42a None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:42.124317 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f0484-05b6-4dbf-b8ec-317c6c40b42a None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:42.124590 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f0484-05b6-4dbf-b8ec-317c6c40b42a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:42.125438 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5f0484-05b6-4dbf-b8ec-317c6c40b42a None admin] RBAC: Adding query filter params (group.id=7a6d9470aae44aa9aa3f20c238c5a421, role.id=1c7adeca27524e258f7ddd2c107e43e1, scope.project.id=e67bbf3bcff7420690a69b58a6241244) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:42.125751 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5f0484-05b6-4dbf-b8ec-317c6c40b42a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:42.128567 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5f0484-05b6-4dbf-b8ec-317c6c40b42a None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:42.135766 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 178/359] 10.222.0.26 () {56 vars in 1376 bytes} [Thu Jun 23 01:43:41 2022] GET /identity/v3/role_assignments?group.id=7a6d9470aae44aa9aa3f20c238c5a421&scope.project.id=e67bbf3bcff7420690a69b58a6241244&role.id=1c7adeca27524e258f7ddd2c107e43e1 => generated 260 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:42.136869 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3de84-95d1-428d-9e18-6fc91ba67329 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:42.137151 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3de84-95d1-428d-9e18-6fc91ba67329 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:42.137803 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3de84-95d1-428d-9e18-6fc91ba67329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:42.155766 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-b6b3de84-95d1-428d-9e18-6fc91ba67329 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:42.157000 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-b6b3de84-95d1-428d-9e18-6fc91ba67329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:42.166738 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:42.194662 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25ec5fd-835c-45ab-b8dc-582a912d0931 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:42.197181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25ec5fd-835c-45ab-b8dc-582a912d0931 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:42.256887 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 179/360] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:42.260193 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b3de84-95d1-428d-9e18-6fc91ba67329 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:42.305757 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:42.353967 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 182/361] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:43:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 01:43:42.356899 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8114f78-d065-4f03-89b9-9bdf841d2262 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:42.357185 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8114f78-d065-4f03-89b9-9bdf841d2262 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:42.357459 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8114f78-d065-4f03-89b9-9bdf841d2262 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:42.359793 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 183/362] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:42.360353 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8114f78-d065-4f03-89b9-9bdf841d2262 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:42.393850 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b3de84-95d1-428d-9e18-6fc91ba67329 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:42.429900 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f76b5-30a1-4038-bb4d-1609171a1f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:42.431140 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f76b5-30a1-4038-bb4d-1609171a1f7f None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:42.431430 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f76b5-30a1-4038-bb4d-1609171a1f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:42.431702 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f76b5-30a1-4038-bb4d-1609171a1f7f None admin] PATH_INFO: `/v3/users/10391bebe4494b069bcaa54cadd78f55` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:42.432911 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65f76b5-30a1-4038-bb4d-1609171a1f7f None admin] RBAC: Authorizing `identity:get_user(user_id=10391bebe4494b069bcaa54cadd78f55)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:42.434382 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65f76b5-30a1-4038-bb4d-1609171a1f7f None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:42.436456 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 180/363] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:43:42 2022] GET /identity/v3/users/10391bebe4494b069bcaa54cadd78f55 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:42.437241 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f76b5-30a1-4038-bb4d-1609171a1f7f None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:42.468829 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:42.500161 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8114f78-d065-4f03-89b9-9bdf841d2262 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:42.546036 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925f7c11-2422-4d9c-9cfa-fdccc065773a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:42.547877 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f7c11-2422-4d9c-9cfa-fdccc065773a None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:42.548485 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f7c11-2422-4d9c-9cfa-fdccc065773a None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:42.549009 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f7c11-2422-4d9c-9cfa-fdccc065773a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:42.553647 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f76b5-30a1-4038-bb4d-1609171a1f7f None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:42.557397 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:42.565636 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925f7c11-2422-4d9c-9cfa-fdccc065773a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:42.576883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-925f7c11-2422-4d9c-9cfa-fdccc065773a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 01:43:42.576883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:42.576883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:42.576883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:42.576883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:42.576883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:42.576883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:42.576883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:42.576883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:42.576883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:42.576883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:42.576883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:42.576883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:42.576883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:42.576883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:42.576883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:42.578025 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:42.578025 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:42.578025 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:42.578025 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:42.578025 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:42.578025 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:42.578025 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:42.578025 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:42.578025 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:42.578025 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:42.578025 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:42.578025 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:42.578025 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:42.578025 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:42.578025 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:42.578025 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:42.579047 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:42.579047 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:42.579047 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:42.579047 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:42.579047 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:42.579047 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:42.579047 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:42.579047 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:42.579047 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:42.579047 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:42.579047 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 01:43:42.579047 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:43:42.581513 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 184/364] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Jun 23 01:43:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:43:42.583874 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270f26b8-a25a-4887-a095-5943af3c0448 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:42.584322 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270f26b8-a25a-4887-a095-5943af3c0448 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:42.584738 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270f26b8-a25a-4887-a095-5943af3c0448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:42.599142 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf569fd-7843-404b-8d74-f49e4416bc01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:42.600169 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf569fd-7843-404b-8d74-f49e4416bc01 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:42.600545 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf569fd-7843-404b-8d74-f49e4416bc01 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:42.601122 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf569fd-7843-404b-8d74-f49e4416bc01 None admin] PATH_INFO: `/v3/projects/9db43d2e143f49efbddf812fd9d03c69` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:42.602309 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf569fd-7843-404b-8d74-f49e4416bc01 None admin] RBAC: Authorizing `identity:get_project(project_id=9db43d2e143f49efbddf812fd9d03c69)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:42.603947 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf569fd-7843-404b-8d74-f49e4416bc01 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:42.606785 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-270f26b8-a25a-4887-a095-5943af3c0448 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:42.607823 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 181/365] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:43:42 2022] GET /identity/v3/projects/9db43d2e143f49efbddf812fd9d03c69 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:42.608521 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-270f26b8-a25a-4887-a095-5943af3c0448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:42.608637 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf569fd-7843-404b-8d74-f49e4416bc01 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:42.694665 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 185/366] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:42.696137 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270f26b8-a25a-4887-a095-5943af3c0448 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:42.760648 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf569fd-7843-404b-8d74-f49e4416bc01 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:42.779971 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:42.812456 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481e64ed-02a4-448a-9e8c-f4fd5a9a20ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:42.813671 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e64ed-02a4-448a-9e8c-f4fd5a9a20ff None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:42.813958 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e64ed-02a4-448a-9e8c-f4fd5a9a20ff None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:42.814233 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e64ed-02a4-448a-9e8c-f4fd5a9a20ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:42.815382 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481e64ed-02a4-448a-9e8c-f4fd5a9a20ff None admin] RBAC: Adding query filter params (role.id=d159e6edeef94e418a4616504e214063, user.id=10391bebe4494b069bcaa54cadd78f55, scope.system=all) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:42.815705 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481e64ed-02a4-448a-9e8c-f4fd5a9a20ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:42.820920 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481e64ed-02a4-448a-9e8c-f4fd5a9a20ff None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:42.837512 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 182/367] 10.222.0.26 () {56 vars in 1308 bytes} [Thu Jun 23 01:43:42 2022] GET /identity/v3/role_assignments?user.id=10391bebe4494b069bcaa54cadd78f55&scope.system=all&role.id=d159e6edeef94e418a4616504e214063 => generated 510 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:42.838355 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481e64ed-02a4-448a-9e8c-f4fd5a9a20ff None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:42.839874 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270f26b8-a25a-4887-a095-5943af3c0448 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:42.900645 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:42.917825 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423d4177-051a-4441-9be6-51240e17f9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:42.918525 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d4177-051a-4441-9be6-51240e17f9d2 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:42.918807 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d4177-051a-4441-9be6-51240e17f9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:42.919084 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d4177-051a-4441-9be6-51240e17f9d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:42.919918 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423d4177-051a-4441-9be6-51240e17f9d2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:42.920226 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423d4177-051a-4441-9be6-51240e17f9d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:42.922056 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423d4177-051a-4441-9be6-51240e17f9d2 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:42.941341 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 186/368] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:43:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:42.943930 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423d4177-051a-4441-9be6-51240e17f9d2 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:43.034692 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481e64ed-02a4-448a-9e8c-f4fd5a9a20ff None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:43.076404 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7bc780-862a-47f2-af61-bf6350191fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:43.077364 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7bc780-862a-47f2-af61-bf6350191fc2 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:43.077649 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7bc780-862a-47f2-af61-bf6350191fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:43.077953 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7bc780-862a-47f2-af61-bf6350191fc2 None admin] PATH_INFO: `/v3/groups/7a6d9470aae44aa9aa3f20c238c5a421` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:43.079354 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7bc780-862a-47f2-af61-bf6350191fc2 None admin] RBAC: Authorizing `identity:get_group(group_id=7a6d9470aae44aa9aa3f20c238c5a421)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:43.081189 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7bc780-862a-47f2-af61-bf6350191fc2 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:43.083884 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 183/369] 10.222.0.26 () {56 vars in 1180 bytes} [Thu Jun 23 01:43:42 2022] GET /identity/v3/groups/7a6d9470aae44aa9aa3f20c238c5a421 => generated 226 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:43.084767 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7bc780-862a-47f2-af61-bf6350191fc2 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:43.097059 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423d4177-051a-4441-9be6-51240e17f9d2 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:43.130301 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:43.138671 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e296d-53f3-4f90-a249-bdcf3bc6c4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:43.139580 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e296d-53f3-4f90-a249-bdcf3bc6c4ce None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:43.139869 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e296d-53f3-4f90-a249-bdcf3bc6c4ce None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:43.140162 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e296d-53f3-4f90-a249-bdcf3bc6c4ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:43.153383 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2e296d-53f3-4f90-a249-bdcf3bc6c4ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:43.165326 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-0c2e296d-53f3-4f90-a249-bdcf3bc6c4ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:43:43.165326 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:43.165326 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:43.165326 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:43.165326 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:43.165326 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:43.165326 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:43.165326 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:43.165326 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:43.165326 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:43.165326 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:43.165326 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:43.165326 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:43.165326 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:43.165326 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:43.165326 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:43.166215 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:43.166215 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:43.166215 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:43.166215 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:43.166215 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:43.166215 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:43.166215 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:43.166215 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:43.166215 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:43.166215 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:43.166215 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:43.166215 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:43.166215 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:43.166215 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:43.166215 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:43.166215 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:43.167426 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:43.167426 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:43.167426 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:43.167426 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:43.167426 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:43.167426 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:43.167426 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:43.167426 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:43.167426 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:43.167426 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:43.167426 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:43:43.167426 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:43:43.167426 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 187/370] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jun 23 01:43:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:43:43.168300 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e296d-53f3-4f90-a249-bdcf3bc6c4ce None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:43.216584 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:43.225100 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7bc780-862a-47f2-af61-bf6350191fc2 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:43.270717 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bbf752-4b5d-4097-b21f-5f0aaf2fe68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:43.271416 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bbf752-4b5d-4097-b21f-5f0aaf2fe68a None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:43.272065 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bbf752-4b5d-4097-b21f-5f0aaf2fe68a None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:43.272344 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bbf752-4b5d-4097-b21f-5f0aaf2fe68a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:43.300395 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e296d-53f3-4f90-a249-bdcf3bc6c4ce None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:43.335561 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed2197a-e545-4f52-a58f-edc781cbf362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:43.336589 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2197a-e545-4f52-a58f-edc781cbf362 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:43.337063 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2197a-e545-4f52-a58f-edc781cbf362 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:43.337535 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2197a-e545-4f52-a58f-edc781cbf362 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:43.338579 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed2197a-e545-4f52-a58f-edc781cbf362 None admin] RBAC: Adding query filter params (group.id=7a6d9470aae44aa9aa3f20c238c5a421, role.id=9021e3c4e3384a309951be4cc1d852e5, scope.project.id=9db43d2e143f49efbddf812fd9d03c69) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:43.339084 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed2197a-e545-4f52-a58f-edc781cbf362 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:43.340768 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed2197a-e545-4f52-a58f-edc781cbf362 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:43.347005 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 188/371] 10.222.0.26 () {56 vars in 1376 bytes} [Thu Jun 23 01:43:43 2022] GET /identity/v3/role_assignments?group.id=7a6d9470aae44aa9aa3f20c238c5a421&scope.project.id=9db43d2e143f49efbddf812fd9d03c69&role.id=9021e3c4e3384a309951be4cc1d852e5 => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:43.348035 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed2197a-e545-4f52-a58f-edc781cbf362 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:43.371339 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bbf752-4b5d-4097-b21f-5f0aaf2fe68a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:43.373188 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bbf752-4b5d-4097-b21f-5f0aaf2fe68a None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:43.477772 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:43.565522 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed2197a-e545-4f52-a58f-edc781cbf362 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:43.583113 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:43.594980 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 184/372] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:43:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 01:43:43.602033 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bbf752-4b5d-4097-b21f-5f0aaf2fe68a None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:43.609946 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577aee0a-4234-4f3e-ab8d-20aa28c96230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:43.610959 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577aee0a-4234-4f3e-ab8d-20aa28c96230 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:43.611475 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577aee0a-4234-4f3e-ab8d-20aa28c96230 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:43.611955 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577aee0a-4234-4f3e-ab8d-20aa28c96230 None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:43.612898 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577aee0a-4234-4f3e-ab8d-20aa28c96230 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:43.613404 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577aee0a-4234-4f3e-ab8d-20aa28c96230 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:43.615102 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577aee0a-4234-4f3e-ab8d-20aa28c96230 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:43.629976 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 189/373] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:43:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:43.634351 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577aee0a-4234-4f3e-ab8d-20aa28c96230 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:43.777644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bbf752-4b5d-4097-b21f-5f0aaf2fe68a None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:43.822700 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d96648-1d8a-4f66-854e-ec94fae4ecaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:43.823980 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d96648-1d8a-4f66-854e-ec94fae4ecaa None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:43.824465 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d96648-1d8a-4f66-854e-ec94fae4ecaa None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:43.824932 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d96648-1d8a-4f66-854e-ec94fae4ecaa None admin] PATH_INFO: `/v3/projects/9db43d2e143f49efbddf812fd9d03c69` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:43.826322 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d96648-1d8a-4f66-854e-ec94fae4ecaa None admin] RBAC: Authorizing `identity:get_project(project_id=9db43d2e143f49efbddf812fd9d03c69)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:43.828012 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d96648-1d8a-4f66-854e-ec94fae4ecaa None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:43.830233 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 185/374] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:43:43 2022] GET /identity/v3/projects/9db43d2e143f49efbddf812fd9d03c69 => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:43:43.836277 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d96648-1d8a-4f66-854e-ec94fae4ecaa None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:43.848632 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577aee0a-4234-4f3e-ab8d-20aa28c96230 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:43.869076 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:43.888956 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3233c2a-3ae1-401b-a427-3014cb4d7983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:43.889722 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3233c2a-3ae1-401b-a427-3014cb4d7983 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:43.890064 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3233c2a-3ae1-401b-a427-3014cb4d7983 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:43.890399 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3233c2a-3ae1-401b-a427-3014cb4d7983 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:43.933580 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:43.967052 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3233c2a-3ae1-401b-a427-3014cb4d7983 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:43.968714 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3233c2a-3ae1-401b-a427-3014cb4d7983 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:43.982722 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d96648-1d8a-4f66-854e-ec94fae4ecaa None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:44.024086 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2642bcb8-b92a-4eaa-adab-6b5f73ad5424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:44.025301 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642bcb8-b92a-4eaa-adab-6b5f73ad5424 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:44.025761 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642bcb8-b92a-4eaa-adab-6b5f73ad5424 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:44.026213 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642bcb8-b92a-4eaa-adab-6b5f73ad5424 None admin] PATH_INFO: `/v3/groups/7a6d9470aae44aa9aa3f20c238c5a421` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:44.027623 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2642bcb8-b92a-4eaa-adab-6b5f73ad5424 None admin] RBAC: Authorizing `identity:get_group(group_id=7a6d9470aae44aa9aa3f20c238c5a421)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:44.029273 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2642bcb8-b92a-4eaa-adab-6b5f73ad5424 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:44.031880 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 186/375] 10.222.0.26 () {56 vars in 1180 bytes} [Thu Jun 23 01:43:43 2022] GET /identity/v3/groups/7a6d9470aae44aa9aa3f20c238c5a421 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:44.039522 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2642bcb8-b92a-4eaa-adab-6b5f73ad5424 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:44.151102 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 190/376] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:43:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 01:43:44.152481 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6968088-8c56-49fa-b27f-b64b9cf3a473 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:44.152771 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6968088-8c56-49fa-b27f-b64b9cf3a473 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:44.153044 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6968088-8c56-49fa-b27f-b64b9cf3a473 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:44.154448 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 191/377] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:44.163704 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc8ff1-113a-4a47-90ec-456c1a3e86ca None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:44.163991 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc8ff1-113a-4a47-90ec-456c1a3e86ca None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:44.164265 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc8ff1-113a-4a47-90ec-456c1a3e86ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:44.187781 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-2fbc8ff1-113a-4a47-90ec-456c1a3e86ca None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:44.189345 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-2fbc8ff1-113a-4a47-90ec-456c1a3e86ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:44.190202 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2642bcb8-b92a-4eaa-adab-6b5f73ad5424 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:44.240103 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a36805-2555-497c-bdb3-f4bba6d197af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:44.240103 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a36805-2555-497c-bdb3-f4bba6d197af None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:44.240103 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a36805-2555-497c-bdb3-f4bba6d197af None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:44.240103 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a36805-2555-497c-bdb3-f4bba6d197af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:44.240925 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a36805-2555-497c-bdb3-f4bba6d197af None admin] RBAC: Adding query filter params (group.id=7a6d9470aae44aa9aa3f20c238c5a421, role.id=1c7adeca27524e258f7ddd2c107e43e1, scope.project.id=9db43d2e143f49efbddf812fd9d03c69) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:44.241239 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a36805-2555-497c-bdb3-f4bba6d197af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:44.242698 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a36805-2555-497c-bdb3-f4bba6d197af None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:44.245496 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:44.250100 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:44.261925 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 187/378] 10.222.0.26 () {56 vars in 1376 bytes} [Thu Jun 23 01:43:44 2022] GET /identity/v3/role_assignments?group.id=7a6d9470aae44aa9aa3f20c238c5a421&scope.project.id=9db43d2e143f49efbddf812fd9d03c69&role.id=1c7adeca27524e258f7ddd2c107e43e1 => generated 260 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:43:44.263005 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a36805-2555-497c-bdb3-f4bba6d197af None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:44.312124 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 192/379] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 01:43:44.342278 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbc8ff1-113a-4a47-90ec-456c1a3e86ca None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:44.433553 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a36805-2555-497c-bdb3-f4bba6d197af None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:44.484367 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604192a3-57e8-4f04-b122-94c691998153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:44.485308 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604192a3-57e8-4f04-b122-94c691998153 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:44.486048 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604192a3-57e8-4f04-b122-94c691998153 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:44.486322 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604192a3-57e8-4f04-b122-94c691998153 None admin] PATH_INFO: `/v3/projects/cc3d5c56f68947b3b757a0c2a92187ac` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:44.487878 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604192a3-57e8-4f04-b122-94c691998153 None admin] RBAC: Authorizing `identity:get_project(project_id=cc3d5c56f68947b3b757a0c2a92187ac)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:44.489880 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604192a3-57e8-4f04-b122-94c691998153 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:44.491996 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 188/380] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:43:44 2022] GET /identity/v3/projects/cc3d5c56f68947b3b757a0c2a92187ac => generated 344 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:44.497438 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604192a3-57e8-4f04-b122-94c691998153 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:44.507735 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbc8ff1-113a-4a47-90ec-456c1a3e86ca None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:44.549855 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c5017f-4565-472c-8305-7d3ad3a89f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:44.550707 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5017f-4565-472c-8305-7d3ad3a89f16 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:44.550994 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5017f-4565-472c-8305-7d3ad3a89f16 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:44.551349 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5017f-4565-472c-8305-7d3ad3a89f16 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:44.563182 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c5017f-4565-472c-8305-7d3ad3a89f16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:44.571378 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-87c5017f-4565-472c-8305-7d3ad3a89f16 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 01:43:44.571378 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:44.571378 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:44.571378 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:44.571378 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:44.571378 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:44.571378 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:44.571378 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:44.571378 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:44.571378 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:44.571378 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:44.571378 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:44.571378 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:44.571378 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:44.571378 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:44.571378 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:44.572257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:44.572257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:44.572257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:44.572257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:44.572257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:44.572257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:44.572257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:44.572257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:44.572257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:44.572257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:44.572257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:44.572257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:44.572257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:44.572257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:44.572257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:44.572257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:44.573060 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:44.573060 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:44.573060 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:44.573060 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:44.573060 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:44.573060 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:44.573060 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:44.573060 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:44.573060 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:44.573060 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:44.573060 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 01:43:44.573060 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:43:44.573060 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 193/381] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:43:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:43:44.577005 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:44.582073 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c5017f-4565-472c-8305-7d3ad3a89f16 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:44.594319 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:44.627117 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604192a3-57e8-4f04-b122-94c691998153 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:44.674737 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91286ff2-da81-4229-99c1-10c5146eadfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:44.675755 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91286ff2-da81-4229-99c1-10c5146eadfa None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:44.676108 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91286ff2-da81-4229-99c1-10c5146eadfa None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:44.676554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91286ff2-da81-4229-99c1-10c5146eadfa None admin] PATH_INFO: `/v3/groups/f0e4bbd719ad4ad9baf7562c7fccc891` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:44.682299 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91286ff2-da81-4229-99c1-10c5146eadfa None admin] RBAC: Authorizing `identity:get_group(group_id=f0e4bbd719ad4ad9baf7562c7fccc891)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:44.683897 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91286ff2-da81-4229-99c1-10c5146eadfa None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:44.685865 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 189/382] 10.222.0.26 () {56 vars in 1180 bytes} [Thu Jun 23 01:43:44 2022] GET /identity/v3/groups/f0e4bbd719ad4ad9baf7562c7fccc891 => generated 229 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:44.691922 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91286ff2-da81-4229-99c1-10c5146eadfa None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:44.717291 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c5017f-4565-472c-8305-7d3ad3a89f16 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:44.753892 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ba3c8-a39c-4843-b8ea-6d9cedc9111f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:44.754612 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ba3c8-a39c-4843-b8ea-6d9cedc9111f None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:44.754892 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ba3c8-a39c-4843-b8ea-6d9cedc9111f None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:44.755161 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ba3c8-a39c-4843-b8ea-6d9cedc9111f None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:44.755939 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748ba3c8-a39c-4843-b8ea-6d9cedc9111f None admin] RBAC: Adding query filter params (name=member) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:44.756242 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748ba3c8-a39c-4843-b8ea-6d9cedc9111f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:44.757632 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748ba3c8-a39c-4843-b8ea-6d9cedc9111f None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:44.766799 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 194/383] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jun 23 01:43:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:43:44.768180 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef136a-7269-485c-a1ec-fa50f0138895 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:44.768465 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef136a-7269-485c-a1ec-fa50f0138895 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:44.768738 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef136a-7269-485c-a1ec-fa50f0138895 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:44.770379 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 195/384] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:44.776122 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ef136a-7269-485c-a1ec-fa50f0138895 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:44.804248 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91286ff2-da81-4229-99c1-10c5146eadfa None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:44.834788 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:44.843074 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1709cb3-5ed2-42fc-a101-1538064456dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:44.844118 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1709cb3-5ed2-42fc-a101-1538064456dd None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:44.844406 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1709cb3-5ed2-42fc-a101-1538064456dd None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:44.844955 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1709cb3-5ed2-42fc-a101-1538064456dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:44.845857 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1709cb3-5ed2-42fc-a101-1538064456dd None admin] RBAC: Adding query filter params (group.id=f0e4bbd719ad4ad9baf7562c7fccc891, role.id=7b20d65f49ee4a48894b6c060bb5d580, scope.project.id=cc3d5c56f68947b3b757a0c2a92187ac) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:44.846180 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1709cb3-5ed2-42fc-a101-1538064456dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:44.847829 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1709cb3-5ed2-42fc-a101-1538064456dd None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:44.855157 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 190/385] 10.222.0.26 () {56 vars in 1376 bytes} [Thu Jun 23 01:43:44 2022] GET /identity/v3/role_assignments?group.id=f0e4bbd719ad4ad9baf7562c7fccc891&scope.project.id=cc3d5c56f68947b3b757a0c2a92187ac&role.id=7b20d65f49ee4a48894b6c060bb5d580 => generated 260 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:43:44.855608 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:44.856982 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eca097-6351-4a71-b9d2-d19252d4bece None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:44.857278 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eca097-6351-4a71-b9d2-d19252d4bece None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:44.857566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eca097-6351-4a71-b9d2-d19252d4bece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:44.881743 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-45eca097-6351-4a71-b9d2-d19252d4bece None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:44.882976 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-45eca097-6351-4a71-b9d2-d19252d4bece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:44.898428 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ef136a-7269-485c-a1ec-fa50f0138895 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:44.936794 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3704e5-76ec-412b-b5f6-a5b998617f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:44.937586 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3704e5-76ec-412b-b5f6-a5b998617f11 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:44.937868 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3704e5-76ec-412b-b5f6-a5b998617f11 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:44.938136 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3704e5-76ec-412b-b5f6-a5b998617f11 None admin] PATH_INFO: `/v3/groups/7a6d9470aae44aa9aa3f20c238c5a421` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:44.940546 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3704e5-76ec-412b-b5f6-a5b998617f11 None admin] RBAC: Authorizing `identity:get_group(group_id=7a6d9470aae44aa9aa3f20c238c5a421)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:44.942076 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3704e5-76ec-412b-b5f6-a5b998617f11 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:44.944397 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 196/386] 10.222.0.26 () {56 vars in 1180 bytes} [Thu Jun 23 01:43:44 2022] GET /identity/v3/groups/7a6d9470aae44aa9aa3f20c238c5a421 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:44.949619 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3704e5-76ec-412b-b5f6-a5b998617f11 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:44.967570 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 191/387] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:44.996114 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eca097-6351-4a71-b9d2-d19252d4bece None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:45.067598 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3704e5-76ec-412b-b5f6-a5b998617f11 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:45.116649 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f34b3-f427-4027-9654-7b7dc6f7f655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:45.117537 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f34b3-f427-4027-9654-7b7dc6f7f655 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:45.117813 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f34b3-f427-4027-9654-7b7dc6f7f655 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:45.118086 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f34b3-f427-4027-9654-7b7dc6f7f655 None admin] PATH_INFO: `/v3/projects/e67bbf3bcff7420690a69b58a6241244` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:45.124134 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72f34b3-f427-4027-9654-7b7dc6f7f655 None admin] RBAC: Authorizing `identity:get_project(project_id=e67bbf3bcff7420690a69b58a6241244)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:45.125121 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eca097-6351-4a71-b9d2-d19252d4bece None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:45.125640 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72f34b3-f427-4027-9654-7b7dc6f7f655 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:45.127412 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 197/388] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:43:44 2022] GET /identity/v3/projects/e67bbf3bcff7420690a69b58a6241244 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:43:45.133325 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f34b3-f427-4027-9654-7b7dc6f7f655 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:45.142324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:45.154186 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:45.166348 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fdd28b-8447-47d6-960c-b18659421d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:45.167464 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fdd28b-8447-47d6-960c-b18659421d2b None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:45.167964 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fdd28b-8447-47d6-960c-b18659421d2b None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:45.168389 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fdd28b-8447-47d6-960c-b18659421d2b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:45.180101 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fdd28b-8447-47d6-960c-b18659421d2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:45.189251 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-64fdd28b-8447-47d6-960c-b18659421d2b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 01:43:45.189251 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:45.189251 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:45.189251 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:45.189251 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:45.189251 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:45.189251 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:45.189251 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:45.189251 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:45.189251 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:45.189251 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:45.189251 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:45.189251 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:45.189251 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:45.189251 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:45.189251 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:45.190357 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:45.190357 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:45.190357 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:45.190357 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:45.190357 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:45.190357 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:45.190357 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:45.190357 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:45.190357 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:45.190357 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:45.190357 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:45.190357 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:45.190357 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:45.190357 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:45.190357 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:45.190357 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:45.191358 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:45.191358 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:45.191358 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:45.191358 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:45.191358 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:45.191358 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:45.191358 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:45.191358 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:45.191358 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:45.191358 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:45.191358 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 01:43:45.191358 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:43:45.193718 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 192/389] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Jun 23 01:43:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:43:45.199370 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fdd28b-8447-47d6-960c-b18659421d2b None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:45.246013 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f34b3-f427-4027-9654-7b7dc6f7f655 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:45.281795 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dee66f-e731-4cc2-b84b-780f851bdf3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:45.284022 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dee66f-e731-4cc2-b84b-780f851bdf3a None admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:45.284300 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dee66f-e731-4cc2-b84b-780f851bdf3a None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:45.284579 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dee66f-e731-4cc2-b84b-780f851bdf3a None admin] PATH_INFO: `/v3/projects/e67bbf3bcff7420690a69b58a6241244/groups/7a6d9470aae44aa9aa3f20c238c5a421/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:45.286684 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dee66f-e731-4cc2-b84b-780f851bdf3a None admin] RBAC: Authorizing `identity:create_grant(project_id=e67bbf3bcff7420690a69b58a6241244, group_id=7a6d9470aae44aa9aa3f20c238c5a421, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:45.288162 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dee66f-e731-4cc2-b84b-780f851bdf3a None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:45.306759 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 198/390] 10.222.0.26 () {58 vars in 1442 bytes} [Thu Jun 23 01:43:45 2022] PUT /identity/v3/projects/e67bbf3bcff7420690a69b58a6241244/groups/7a6d9470aae44aa9aa3f20c238c5a421/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 01:43:45.313056 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fdd28b-8447-47d6-960c-b18659421d2b None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:45.356007 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f08e896-9a31-407e-80d4-474ee5782abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:45.357034 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08e896-9a31-407e-80d4-474ee5782abc None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:45.357554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08e896-9a31-407e-80d4-474ee5782abc None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:45.358044 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08e896-9a31-407e-80d4-474ee5782abc None admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:45.358954 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f08e896-9a31-407e-80d4-474ee5782abc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:45.359433 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f08e896-9a31-407e-80d4-474ee5782abc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:45.361186 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f08e896-9a31-407e-80d4-474ee5782abc None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:45.371779 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 193/391] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:43:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:43:45.378885 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dee66f-e731-4cc2-b84b-780f851bdf3a None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:45.466866 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:45.511151 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dee66f-e731-4cc2-b84b-780f851bdf3a None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:45.545066 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6835093e-f11a-4276-b132-2b794017b2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:45.545931 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6835093e-f11a-4276-b132-2b794017b2ed None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:45.546205 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6835093e-f11a-4276-b132-2b794017b2ed None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:45.546473 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6835093e-f11a-4276-b132-2b794017b2ed None admin] PATH_INFO: `/v3/groups/7a6d9470aae44aa9aa3f20c238c5a421` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:45.547725 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6835093e-f11a-4276-b132-2b794017b2ed None admin] RBAC: Authorizing `identity:get_group(group_id=7a6d9470aae44aa9aa3f20c238c5a421)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:45.549164 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6835093e-f11a-4276-b132-2b794017b2ed None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:45.551133 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 199/392] 10.222.0.26 () {56 vars in 1180 bytes} [Thu Jun 23 01:43:45 2022] GET /identity/v3/groups/7a6d9470aae44aa9aa3f20c238c5a421 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:45.556658 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f08e896-9a31-407e-80d4-474ee5782abc None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:45.635803 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:45.701025 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f08e896-9a31-407e-80d4-474ee5782abc None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:45.752002 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebff99e2-26cc-40f5-91c3-25ee68d1d56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:45.753208 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff99e2-26cc-40f5-91c3-25ee68d1d56b None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:45.753691 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff99e2-26cc-40f5-91c3-25ee68d1d56b None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:45.754223 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff99e2-26cc-40f5-91c3-25ee68d1d56b None admin] PATH_INFO: `/v3/projects/e67bbf3bcff7420690a69b58a6241244` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:45.755747 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebff99e2-26cc-40f5-91c3-25ee68d1d56b None admin] RBAC: Authorizing `identity:get_project(project_id=e67bbf3bcff7420690a69b58a6241244)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:45.757431 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebff99e2-26cc-40f5-91c3-25ee68d1d56b None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:45.759899 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 194/393] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:43:45 2022] GET /identity/v3/projects/e67bbf3bcff7420690a69b58a6241244 => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:45.765556 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6835093e-f11a-4276-b132-2b794017b2ed None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:45.914454 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6835093e-f11a-4276-b132-2b794017b2ed None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:45.954281 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:45.959794 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0089423-54f6-4019-9505-47cd9abb0871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:45.961134 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0089423-54f6-4019-9505-47cd9abb0871 None admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:45.961653 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0089423-54f6-4019-9505-47cd9abb0871 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:45.962279 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0089423-54f6-4019-9505-47cd9abb0871 None admin] PATH_INFO: `/v3/projects/e67bbf3bcff7420690a69b58a6241244/groups/7a6d9470aae44aa9aa3f20c238c5a421/roles/1c7adeca27524e258f7ddd2c107e43e1` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:45.964572 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0089423-54f6-4019-9505-47cd9abb0871 None admin] RBAC: Authorizing `identity:create_grant(project_id=e67bbf3bcff7420690a69b58a6241244, group_id=7a6d9470aae44aa9aa3f20c238c5a421, role_id=1c7adeca27524e258f7ddd2c107e43e1)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:45.966045 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0089423-54f6-4019-9505-47cd9abb0871 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:45.983006 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 200/394] 10.222.0.26 () {58 vars in 1442 bytes} [Thu Jun 23 01:43:45 2022] PUT /identity/v3/projects/e67bbf3bcff7420690a69b58a6241244/groups/7a6d9470aae44aa9aa3f20c238c5a421/roles/1c7adeca27524e258f7ddd2c107e43e1 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:43:46.384166 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5a1f51-df60-4e58-9a22-df8acd4f14f6 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:46.385125 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5a1f51-df60-4e58-9a22-df8acd4f14f6 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:46.385630 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5a1f51-df60-4e58-9a22-df8acd4f14f6 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:46.387529 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 195/395] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:46.392551 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131bedef-de41-4e15-9fde-e65f49bbfcfc None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:46.393001 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131bedef-de41-4e15-9fde-e65f49bbfcfc None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:46.393084 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131bedef-de41-4e15-9fde-e65f49bbfcfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:46.415384 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-131bedef-de41-4e15-9fde-e65f49bbfcfc None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:46.416570 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-131bedef-de41-4e15-9fde-e65f49bbfcfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:46.492106 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 201/396] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:46.515619 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5a1f51-df60-4e58-9a22-df8acd4f14f6 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:46.671096 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5a1f51-df60-4e58-9a22-df8acd4f14f6 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:46.692400 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:46.714193 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5eaf3b-876b-43fb-a80c-43c32a871d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:46.714994 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5eaf3b-876b-43fb-a80c-43c32a871d08 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:46.715298 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5eaf3b-876b-43fb-a80c-43c32a871d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:46.715570 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5eaf3b-876b-43fb-a80c-43c32a871d08 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:46.726388 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5eaf3b-876b-43fb-a80c-43c32a871d08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:46.734099 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-8d5eaf3b-876b-43fb-a80c-43c32a871d08 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 01:43:46.734099 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:46.734099 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:46.734099 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:46.734099 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:46.734099 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:46.734099 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:46.734099 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:46.734099 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:46.734099 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:46.734099 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:46.734099 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:46.734099 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:46.734099 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:46.734099 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:46.734099 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:46.735064 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:46.735064 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:46.735064 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:46.735064 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:46.735064 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:46.735064 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:46.735064 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:46.735064 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:46.735064 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:46.735064 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:46.735064 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:46.735064 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:46.735064 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:46.735064 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:46.735064 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:46.735064 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:46.735859 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:46.735859 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:46.735859 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:46.735859 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:46.735859 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:46.735859 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:46.735859 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:46.735859 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:46.735859 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:46.735859 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:46.735859 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 01:43:46.735859 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:43:46.737661 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 196/397] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:43:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:43:46.743680 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131bedef-de41-4e15-9fde-e65f49bbfcfc None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:46.889566 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131bedef-de41-4e15-9fde-e65f49bbfcfc None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:46.913030 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:46.926059 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4622a0-02fd-4ee3-acb9-54b76a36119f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:46.926840 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4622a0-02fd-4ee3-acb9-54b76a36119f None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:46.927115 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4622a0-02fd-4ee3-acb9-54b76a36119f None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:46.927419 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4622a0-02fd-4ee3-acb9-54b76a36119f None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:46.928322 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4622a0-02fd-4ee3-acb9-54b76a36119f None admin] RBAC: Adding query filter params (name=member) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:46.929003 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4622a0-02fd-4ee3-acb9-54b76a36119f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:46.933038 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4622a0-02fd-4ee3-acb9-54b76a36119f None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:46.942775 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 202/398] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jun 23 01:43:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:46.948050 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5eaf3b-876b-43fb-a80c-43c32a871d08 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:47.078972 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5eaf3b-876b-43fb-a80c-43c32a871d08 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:47.117544 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fce9c6-6b94-4e57-b9a9-245e87a2d7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:47.118411 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fce9c6-6b94-4e57-b9a9-245e87a2d7fc None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:47.118697 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fce9c6-6b94-4e57-b9a9-245e87a2d7fc None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:47.118975 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fce9c6-6b94-4e57-b9a9-245e87a2d7fc None admin] PATH_INFO: `/v3/groups/7a6d9470aae44aa9aa3f20c238c5a421` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:47.120234 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fce9c6-6b94-4e57-b9a9-245e87a2d7fc None admin] RBAC: Authorizing `identity:get_group(group_id=7a6d9470aae44aa9aa3f20c238c5a421)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:47.121700 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fce9c6-6b94-4e57-b9a9-245e87a2d7fc None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:47.123781 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 197/399] 10.222.0.26 () {56 vars in 1180 bytes} [Thu Jun 23 01:43:46 2022] GET /identity/v3/groups/7a6d9470aae44aa9aa3f20c238c5a421 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:47.132615 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4622a0-02fd-4ee3-acb9-54b76a36119f None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:47.199127 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e0d52-14fa-4cb2-bdc2-7cac5444ac7f None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:47.199432 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e0d52-14fa-4cb2-bdc2-7cac5444ac7f None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:47.199681 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e0d52-14fa-4cb2-bdc2-7cac5444ac7f None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:47.201197 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 198/400] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:47.207089 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ff407-ac6c-49f7-895a-c57be2be385a None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:47.207443 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ff407-ac6c-49f7-895a-c57be2be385a None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:47.207808 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ff407-ac6c-49f7-895a-c57be2be385a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:47.239154 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-649ff407-ac6c-49f7-895a-c57be2be385a None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:47.240433 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-649ff407-ac6c-49f7-895a-c57be2be385a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:47.285265 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4622a0-02fd-4ee3-acb9-54b76a36119f None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:47.319244 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:47.365370 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb79399-9829-40d4-a87e-aa31c7ef28a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:47.366612 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb79399-9829-40d4-a87e-aa31c7ef28a5 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:47.368121 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb79399-9829-40d4-a87e-aa31c7ef28a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:47.369270 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb79399-9829-40d4-a87e-aa31c7ef28a5 None admin] PATH_INFO: `/v3/projects/9db43d2e143f49efbddf812fd9d03c69` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:47.371198 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb79399-9829-40d4-a87e-aa31c7ef28a5 None admin] RBAC: Authorizing `identity:get_project(project_id=9db43d2e143f49efbddf812fd9d03c69)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:47.374290 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb79399-9829-40d4-a87e-aa31c7ef28a5 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:47.376615 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 203/401] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:43:47 2022] GET /identity/v3/projects/9db43d2e143f49efbddf812fd9d03c69 => generated 302 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:47.379482 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 199/402] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 01:43:47.384294 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb79399-9829-40d4-a87e-aa31c7ef28a5 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:47.406799 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649ff407-ac6c-49f7-895a-c57be2be385a None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:47.483369 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:47.545217 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb79399-9829-40d4-a87e-aa31c7ef28a5 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:47.581651 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649ff407-ac6c-49f7-895a-c57be2be385a None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:47.593318 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a9f03f-fa75-4670-bde9-07377b173442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:47.594224 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a9f03f-fa75-4670-bde9-07377b173442 None admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:47.594503 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a9f03f-fa75-4670-bde9-07377b173442 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:47.594779 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a9f03f-fa75-4670-bde9-07377b173442 None admin] PATH_INFO: `/v3/projects/9db43d2e143f49efbddf812fd9d03c69/groups/7a6d9470aae44aa9aa3f20c238c5a421/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:47.596788 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a9f03f-fa75-4670-bde9-07377b173442 None admin] RBAC: Authorizing `identity:create_grant(project_id=9db43d2e143f49efbddf812fd9d03c69, group_id=7a6d9470aae44aa9aa3f20c238c5a421, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:47.598307 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a9f03f-fa75-4670-bde9-07377b173442 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:47.625903 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 204/403] 10.222.0.26 () {58 vars in 1442 bytes} [Thu Jun 23 01:43:47 2022] PUT /identity/v3/projects/9db43d2e143f49efbddf812fd9d03c69/groups/7a6d9470aae44aa9aa3f20c238c5a421/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:43:47.628733 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19b237-b9c7-486f-9a66-4ed13aeec4bd None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:47.629246 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19b237-b9c7-486f-9a66-4ed13aeec4bd None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:47.629775 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19b237-b9c7-486f-9a66-4ed13aeec4bd None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:47.632414 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 205/404] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:47.634203 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18c7f1-0b45-4113-b247-31019e5808fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:47.635328 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18c7f1-0b45-4113-b247-31019e5808fa None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:47.635821 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18c7f1-0b45-4113-b247-31019e5808fa None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:47.636336 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18c7f1-0b45-4113-b247-31019e5808fa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:47.638597 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297233e3-8c33-482c-b1a3-0333f4738c27 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:47.639310 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297233e3-8c33-482c-b1a3-0333f4738c27 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:47.641151 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297233e3-8c33-482c-b1a3-0333f4738c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:47.662827 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e18c7f1-0b45-4113-b247-31019e5808fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:47.671547 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-1e18c7f1-0b45-4113-b247-31019e5808fa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 01:43:47.671547 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:47.671547 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:47.671547 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:47.671547 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:47.671547 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:47.671547 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:47.671547 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:47.671547 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:47.671547 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:47.671547 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:47.671547 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:47.671547 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:47.671547 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:47.671547 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:47.671547 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:47.672499 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:47.672499 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:47.672499 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:47.672499 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:47.672499 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:47.672499 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:47.672499 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:47.672499 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:47.672499 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:47.672499 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:47.672499 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:47.672499 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:47.672499 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:47.672499 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:47.672499 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:47.672499 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:47.673270 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:47.673270 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:47.673270 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:47.673270 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:47.673270 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:47.673270 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:47.673270 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:47.673270 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:47.673270 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:47.673270 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:47.673270 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 01:43:47.673270 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:43:47.681773 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-297233e3-8c33-482c-b1a3-0333f4738c27 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:47.683315 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-297233e3-8c33-482c-b1a3-0333f4738c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:47.685610 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 200/405] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Jun 23 01:43:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:43:47.686732 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18c7f1-0b45-4113-b247-31019e5808fa None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:47.730620 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:47.849822 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 206/406] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:47.851088 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ddb3a2-61a6-4c42-b38e-3c1d1f6c95f9 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:47.851406 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ddb3a2-61a6-4c42-b38e-3c1d1f6c95f9 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:47.851692 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ddb3a2-61a6-4c42-b38e-3c1d1f6c95f9 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:47.857496 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18c7f1-0b45-4113-b247-31019e5808fa None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:47.859370 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 207/407] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:47 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:47.864966 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea0f873-24b6-41ce-8c0a-9f166a0c3350 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:47.865366 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea0f873-24b6-41ce-8c0a-9f166a0c3350 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:47.865645 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea0f873-24b6-41ce-8c0a-9f166a0c3350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:47.889425 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-fea0f873-24b6-41ce-8c0a-9f166a0c3350 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:47.890624 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-fea0f873-24b6-41ce-8c0a-9f166a0c3350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:47.899353 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c08be4-c26e-41f1-b588-bde64140dac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:47.900383 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c08be4-c26e-41f1-b588-bde64140dac3 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:47.900840 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c08be4-c26e-41f1-b588-bde64140dac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:47.901311 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c08be4-c26e-41f1-b588-bde64140dac3 None admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:47.902257 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c08be4-c26e-41f1-b588-bde64140dac3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:47.902734 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c08be4-c26e-41f1-b588-bde64140dac3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:47.904511 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c08be4-c26e-41f1-b588-bde64140dac3 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:47.913970 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 201/408] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:43:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:47.919237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c08be4-c26e-41f1-b588-bde64140dac3 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:47.928488 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:47.963916 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 208/409] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:47.971396 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea0f873-24b6-41ce-8c0a-9f166a0c3350 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:48.043600 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:48.050834 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c08be4-c26e-41f1-b588-bde64140dac3 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:48.069621 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea0f873-24b6-41ce-8c0a-9f166a0c3350 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:48.086322 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2612a3-436a-4e2c-9391-fb921ea8da26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:48.087388 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2612a3-436a-4e2c-9391-fb921ea8da26 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:48.087848 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2612a3-436a-4e2c-9391-fb921ea8da26 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:48.088301 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2612a3-436a-4e2c-9391-fb921ea8da26 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:48.099327 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2612a3-436a-4e2c-9391-fb921ea8da26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:48.104620 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1c7383-2cae-457f-9a70-949a95e0f3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:48.105427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1c7383-2cae-457f-9a70-949a95e0f3db None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:48.105713 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1c7383-2cae-457f-9a70-949a95e0f3db None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:48.105984 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1c7383-2cae-457f-9a70-949a95e0f3db None admin] PATH_INFO: `/v3/groups/7a6d9470aae44aa9aa3f20c238c5a421` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:48.107175 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1c7383-2cae-457f-9a70-949a95e0f3db None admin] RBAC: Authorizing `identity:get_group(group_id=7a6d9470aae44aa9aa3f20c238c5a421)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:48.108667 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1c7383-2cae-457f-9a70-949a95e0f3db None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:48.109178 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-de2612a3-436a-4e2c-9391-fb921ea8da26 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:43:48.109178 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:48.109178 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:48.109178 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:48.109178 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:48.109178 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:48.109178 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:48.109178 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:48.109178 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:48.109178 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:48.109178 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:48.109178 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:48.109178 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:48.109178 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:48.109178 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:48.109178 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:48.110037 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:48.110037 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:48.110037 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:48.110037 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:48.110037 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:48.110037 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:48.110037 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:48.110037 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:48.110037 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:48.110037 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:48.110037 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:48.110037 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:48.110037 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:48.110037 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:48.110037 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:48.110037 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:48.110801 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:48.110801 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:48.110801 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:48.110801 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:48.110801 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:48.110801 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:48.110801 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:48.110801 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:48.110801 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:48.110801 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:48.110801 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:43:48.110801 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:43:48.111411 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 209/410] 10.222.0.26 () {56 vars in 1180 bytes} [Thu Jun 23 01:43:47 2022] GET /identity/v3/groups/7a6d9470aae44aa9aa3f20c238c5a421 => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:48.117937 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 202/411] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jun 23 01:43:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:43:48.118080 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1c7383-2cae-457f-9a70-949a95e0f3db None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:48.118609 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2612a3-436a-4e2c-9391-fb921ea8da26 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:48.158980 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:48.223636 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1c7383-2cae-457f-9a70-949a95e0f3db None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:48.235657 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2612a3-436a-4e2c-9391-fb921ea8da26 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:48.258203 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c623a-14c7-4c95-ab6b-dcaf3013eed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:48.258948 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c623a-14c7-4c95-ab6b-dcaf3013eed5 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:48.259254 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c623a-14c7-4c95-ab6b-dcaf3013eed5 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:48.259587 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c623a-14c7-4c95-ab6b-dcaf3013eed5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:48.270554 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c623a-14c7-4c95-ab6b-dcaf3013eed5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:48.274724 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d49b4-e913-4b0a-b683-af5847b4a599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:48.275947 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d49b4-e913-4b0a-b683-af5847b4a599 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:48.276425 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d49b4-e913-4b0a-b683-af5847b4a599 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:48.276946 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d49b4-e913-4b0a-b683-af5847b4a599 None admin] PATH_INFO: `/v3/projects/9db43d2e143f49efbddf812fd9d03c69` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:48.278243 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-e33c623a-14c7-4c95-ab6b-dcaf3013eed5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 01:43:48.278243 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:48.278243 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:48.278243 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:48.278243 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:48.278243 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:48.278243 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:48.278243 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:48.278243 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:48.278243 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:48.278243 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:48.278243 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:48.278243 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:48.278243 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:48.278243 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:48.278243 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:48.279082 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:48.279082 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:48.279082 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:48.279082 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:48.279082 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:48.279082 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:48.279082 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:48.279082 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:48.279082 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:48.279082 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:48.279082 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:48.279082 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:48.279082 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:48.279082 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:48.279082 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:48.279082 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:48.279921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:48.279921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:48.279921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:48.279921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:48.279921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:48.279921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:48.279921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:48.279921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:48.279921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:48.279921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:48.279921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 01:43:48.279921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:43:48.281310 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461d49b4-e913-4b0a-b683-af5847b4a599 None admin] RBAC: Authorizing `identity:get_project(project_id=9db43d2e143f49efbddf812fd9d03c69)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:48.283029 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461d49b4-e913-4b0a-b683-af5847b4a599 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:48.284292 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 210/412] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:43:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:43:48.285115 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c623a-14c7-4c95-ab6b-dcaf3013eed5 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:48.286042 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 203/413] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:43:48 2022] GET /identity/v3/projects/9db43d2e143f49efbddf812fd9d03c69 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:48.287041 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d49b4-e913-4b0a-b683-af5847b4a599 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:48.315734 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:48.388664 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:48.391870 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c623a-14c7-4c95-ab6b-dcaf3013eed5 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:48.400996 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d49b4-e913-4b0a-b683-af5847b4a599 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:48.428943 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66e8b8b-33fc-4701-a9e5-8f1087ccb36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:48.429887 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e8b8b-33fc-4701-a9e5-8f1087ccb36b None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:48.430178 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e8b8b-33fc-4701-a9e5-8f1087ccb36b None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:48.430771 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e8b8b-33fc-4701-a9e5-8f1087ccb36b None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:48.431588 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66e8b8b-33fc-4701-a9e5-8f1087ccb36b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:48.432134 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66e8b8b-33fc-4701-a9e5-8f1087ccb36b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:48.434201 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66e8b8b-33fc-4701-a9e5-8f1087ccb36b None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:48.441504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e66810-8fc1-4010-b1a6-c4f1cb89a1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:48.442825 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e66810-8fc1-4010-b1a6-c4f1cb89a1ac None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:48.443353 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e66810-8fc1-4010-b1a6-c4f1cb89a1ac None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:48.443705 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 211/414] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:43:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:48.448209 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66e8b8b-33fc-4701-a9e5-8f1087ccb36b None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:48.448709 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e66810-8fc1-4010-b1a6-c4f1cb89a1ac None admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:48.449801 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e66810-8fc1-4010-b1a6-c4f1cb89a1ac None admin] RBAC: Adding query filter params (name=member) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:48.450338 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e66810-8fc1-4010-b1a6-c4f1cb89a1ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:48.452165 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e66810-8fc1-4010-b1a6-c4f1cb89a1ac None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:48.467015 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 204/415] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jun 23 01:43:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:48.468251 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e66810-8fc1-4010-b1a6-c4f1cb89a1ac None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:48.557999 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:48.574695 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66e8b8b-33fc-4701-a9e5-8f1087ccb36b None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:48.586022 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e66810-8fc1-4010-b1a6-c4f1cb89a1ac None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:48.611753 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d68cd5c-b635-49a9-9093-b9fc9df5ff80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:48.612805 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d68cd5c-b635-49a9-9093-b9fc9df5ff80 None admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:48.613091 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d68cd5c-b635-49a9-9093-b9fc9df5ff80 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:48.613365 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d68cd5c-b635-49a9-9093-b9fc9df5ff80 None admin] PATH_INFO: `/v3/projects/9db43d2e143f49efbddf812fd9d03c69/groups/7a6d9470aae44aa9aa3f20c238c5a421/roles/1c7adeca27524e258f7ddd2c107e43e1` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:48.615810 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d68cd5c-b635-49a9-9093-b9fc9df5ff80 None admin] RBAC: Authorizing `identity:create_grant(project_id=9db43d2e143f49efbddf812fd9d03c69, group_id=7a6d9470aae44aa9aa3f20c238c5a421, role_id=1c7adeca27524e258f7ddd2c107e43e1)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:48.617376 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d68cd5c-b635-49a9-9093-b9fc9df5ff80 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:48.629390 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303fe16d-4159-4c79-9dd3-adabb620a28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:48.630615 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303fe16d-4159-4c79-9dd3-adabb620a28a None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:48.631110 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303fe16d-4159-4c79-9dd3-adabb620a28a None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:48.631630 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303fe16d-4159-4c79-9dd3-adabb620a28a None admin] PATH_INFO: `/v3/groups/f0e4bbd719ad4ad9baf7562c7fccc891` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:48.639320 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 212/416] 10.222.0.26 () {58 vars in 1442 bytes} [Thu Jun 23 01:43:48 2022] PUT /identity/v3/projects/9db43d2e143f49efbddf812fd9d03c69/groups/7a6d9470aae44aa9aa3f20c238c5a421/roles/1c7adeca27524e258f7ddd2c107e43e1 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:43:48.639320 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d68cd5c-b635-49a9-9093-b9fc9df5ff80 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:48.640159 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303fe16d-4159-4c79-9dd3-adabb620a28a None admin] RBAC: Authorizing `identity:get_group(group_id=f0e4bbd719ad4ad9baf7562c7fccc891)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:48.641943 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303fe16d-4159-4c79-9dd3-adabb620a28a None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:48.644424 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 205/417] 10.222.0.26 () {56 vars in 1180 bytes} [Thu Jun 23 01:43:48 2022] GET /identity/v3/groups/f0e4bbd719ad4ad9baf7562c7fccc891 => generated 229 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:48.651369 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303fe16d-4159-4c79-9dd3-adabb620a28a None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:48.672027 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:48.762798 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d68cd5c-b635-49a9-9093-b9fc9df5ff80 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:48.810825 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a8406-9e58-4f50-922a-b317d980063c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:48.811766 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a8406-9e58-4f50-922a-b317d980063c None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:48.812056 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a8406-9e58-4f50-922a-b317d980063c None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:48.812337 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a8406-9e58-4f50-922a-b317d980063c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:48.816093 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303fe16d-4159-4c79-9dd3-adabb620a28a None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:48.856826 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24597622-2230-4bd8-8784-72e0f3adc745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:48.857694 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24597622-2230-4bd8-8784-72e0f3adc745 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:48.857973 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24597622-2230-4bd8-8784-72e0f3adc745 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:48.858248 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24597622-2230-4bd8-8784-72e0f3adc745 None admin] PATH_INFO: `/v3/projects/cc3d5c56f68947b3b757a0c2a92187ac` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:48.859518 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24597622-2230-4bd8-8784-72e0f3adc745 None admin] RBAC: Authorizing `identity:get_project(project_id=cc3d5c56f68947b3b757a0c2a92187ac)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:48.861941 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24597622-2230-4bd8-8784-72e0f3adc745 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:48.864118 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 206/418] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:43:48 2022] GET /identity/v3/projects/cc3d5c56f68947b3b757a0c2a92187ac => generated 344 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:48.865359 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b6c31-a807-4f27-8708-5cc47ea7074a None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:48.865643 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b6c31-a807-4f27-8708-5cc47ea7074a None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:48.865917 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b6c31-a807-4f27-8708-5cc47ea7074a None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:48.871170 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 207/419] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:48.871170 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b6c31-a807-4f27-8708-5cc47ea7074a None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:48.896740 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:48.935453 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273a8406-9e58-4f50-922a-b317d980063c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:48.937353 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273a8406-9e58-4f50-922a-b317d980063c None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:48.968385 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:48.985732 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b6c31-a807-4f27-8708-5cc47ea7074a None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:49.022237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87822f-7b10-4651-9c16-454b286526e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:49.023482 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87822f-7b10-4651-9c16-454b286526e6 None admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:49.024031 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87822f-7b10-4651-9c16-454b286526e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:49.024627 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87822f-7b10-4651-9c16-454b286526e6 None admin] PATH_INFO: `/v3/projects/cc3d5c56f68947b3b757a0c2a92187ac/groups/f0e4bbd719ad4ad9baf7562c7fccc891/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:49.026796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e87822f-7b10-4651-9c16-454b286526e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=cc3d5c56f68947b3b757a0c2a92187ac, group_id=f0e4bbd719ad4ad9baf7562c7fccc891, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:49.028500 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e87822f-7b10-4651-9c16-454b286526e6 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:49.046325 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 208/420] 10.222.0.26 () {58 vars in 1442 bytes} [Thu Jun 23 01:43:48 2022] PUT /identity/v3/projects/cc3d5c56f68947b3b757a0c2a92187ac/groups/f0e4bbd719ad4ad9baf7562c7fccc891/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:43:49.049885 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d4e01-e4cf-4e98-be3c-fece8a9083e8 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:49.050471 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d4e01-e4cf-4e98-be3c-fece8a9083e8 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:49.052236 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d4e01-e4cf-4e98-be3c-fece8a9083e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:49.081016 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 213/421] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:43:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 01:43:49.083638 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-6e0d4e01-e4cf-4e98-be3c-fece8a9083e8 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:49.085094 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-6e0d4e01-e4cf-4e98-be3c-fece8a9083e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:49.087071 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a8406-9e58-4f50-922a-b317d980063c None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:49.235316 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 209/422] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:49.253868 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d4e01-e4cf-4e98-be3c-fece8a9083e8 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:49.262307 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a8406-9e58-4f50-922a-b317d980063c None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:49.272913 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:49.307606 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:49.323587 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edcb299-e495-472a-93de-b4556cfbf87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:49.325118 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edcb299-e495-472a-93de-b4556cfbf87e None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:49.325607 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edcb299-e495-472a-93de-b4556cfbf87e None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:49.326078 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edcb299-e495-472a-93de-b4556cfbf87e None admin] PATH_INFO: `/v3/projects/e67bbf3bcff7420690a69b58a6241244` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:49.332077 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edcb299-e495-472a-93de-b4556cfbf87e None admin] RBAC: Authorizing `identity:get_project(project_id=e67bbf3bcff7420690a69b58a6241244)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:49.333983 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edcb299-e495-472a-93de-b4556cfbf87e None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:49.336221 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 214/423] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:43:49 2022] GET /identity/v3/projects/e67bbf3bcff7420690a69b58a6241244 => generated 298 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:43:49.341871 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edcb299-e495-472a-93de-b4556cfbf87e None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:49.381840 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d4e01-e4cf-4e98-be3c-fece8a9083e8 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:49.419898 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894dc726-7da8-416a-a738-ae77a4915509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:49.420709 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894dc726-7da8-416a-a738-ae77a4915509 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:49.420988 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894dc726-7da8-416a-a738-ae77a4915509 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:49.421260 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894dc726-7da8-416a-a738-ae77a4915509 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:49.433706 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894dc726-7da8-416a-a738-ae77a4915509 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:49.447498 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-894dc726-7da8-416a-a738-ae77a4915509 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 01:43:49.447498 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:49.447498 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:49.447498 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:49.447498 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:49.447498 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:49.447498 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:49.447498 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:49.447498 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:49.447498 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:49.447498 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:49.447498 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:49.447498 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:49.447498 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:49.447498 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:49.447498 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:49.448834 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:49.448834 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:49.448834 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:49.448834 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:49.448834 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:49.448834 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:49.448834 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:49.448834 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:49.448834 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:49.448834 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:49.448834 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:49.448834 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:49.448834 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:49.448834 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:49.448834 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:49.448834 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:49.450004 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:49.450004 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:49.450004 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:49.450004 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:49.450004 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:49.450004 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:49.450004 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:49.450004 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:49.450004 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:49.450004 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:49.450004 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 01:43:49.450004 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:43:49.450004 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 210/424] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Jun 23 01:43:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:43:49.457291 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894dc726-7da8-416a-a738-ae77a4915509 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:49.551548 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edcb299-e495-472a-93de-b4556cfbf87e None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:49.587653 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8059e4-f1a8-40c2-8d00-1a2fef62bf0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:49.588525 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8059e4-f1a8-40c2-8d00-1a2fef62bf0d None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:49.588806 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8059e4-f1a8-40c2-8d00-1a2fef62bf0d None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:49.589082 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8059e4-f1a8-40c2-8d00-1a2fef62bf0d None admin] PATH_INFO: `/v3/groups/7a6d9470aae44aa9aa3f20c238c5a421` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:49.590295 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8059e4-f1a8-40c2-8d00-1a2fef62bf0d None admin] RBAC: Authorizing `identity:get_group(group_id=7a6d9470aae44aa9aa3f20c238c5a421)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:49.591796 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8059e4-f1a8-40c2-8d00-1a2fef62bf0d None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:49.593751 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 215/425] 10.222.0.26 () {56 vars in 1180 bytes} [Thu Jun 23 01:43:49 2022] GET /identity/v3/groups/7a6d9470aae44aa9aa3f20c238c5a421 => generated 226 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:49.600667 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8059e4-f1a8-40c2-8d00-1a2fef62bf0d None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:49.623570 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:49.682274 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894dc726-7da8-416a-a738-ae77a4915509 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:49.692411 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:49.741205 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8059e4-f1a8-40c2-8d00-1a2fef62bf0d None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:49.761183 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456585ba-e768-4684-8f0c-ed537ff08b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:49.761856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456585ba-e768-4684-8f0c-ed537ff08b54 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:49.762139 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456585ba-e768-4684-8f0c-ed537ff08b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:49.762415 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456585ba-e768-4684-8f0c-ed537ff08b54 None admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:49.763179 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456585ba-e768-4684-8f0c-ed537ff08b54 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:49.763512 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456585ba-e768-4684-8f0c-ed537ff08b54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:49.764914 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456585ba-e768-4684-8f0c-ed537ff08b54 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:49.776896 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 211/426] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:43:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:49.779059 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda8ed7-05b3-4db8-941b-a8bda524bb6d None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:49.779844 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda8ed7-05b3-4db8-941b-a8bda524bb6d None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:49.780419 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda8ed7-05b3-4db8-941b-a8bda524bb6d None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:49.786468 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 212/427] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:49.786468 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bda8ed7-05b3-4db8-941b-a8bda524bb6d None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:49.786648 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731a8374-29db-4e86-8cb2-b8b8fdb39fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:49.786648 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a8374-29db-4e86-8cb2-b8b8fdb39fda None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:49.786648 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a8374-29db-4e86-8cb2-b8b8fdb39fda None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:49.786995 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a8374-29db-4e86-8cb2-b8b8fdb39fda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:49.787809 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731a8374-29db-4e86-8cb2-b8b8fdb39fda None admin] RBAC: Adding query filter params (group.id=7a6d9470aae44aa9aa3f20c238c5a421, role.id=9021e3c4e3384a309951be4cc1d852e5, scope.project.id=e67bbf3bcff7420690a69b58a6241244) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:49.788122 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731a8374-29db-4e86-8cb2-b8b8fdb39fda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:49.789560 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731a8374-29db-4e86-8cb2-b8b8fdb39fda None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:49.800186 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 216/428] 10.222.0.26 () {56 vars in 1376 bytes} [Thu Jun 23 01:43:49 2022] GET /identity/v3/role_assignments?group.id=7a6d9470aae44aa9aa3f20c238c5a421&scope.project.id=e67bbf3bcff7420690a69b58a6241244&role.id=9021e3c4e3384a309951be4cc1d852e5 => generated 611 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:49.801978 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11735ed-a62c-49d9-a5dc-afac102a13a6 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:49.802215 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11735ed-a62c-49d9-a5dc-afac102a13a6 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:49.802495 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11735ed-a62c-49d9-a5dc-afac102a13a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:49.839903 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-e11735ed-a62c-49d9-a5dc-afac102a13a6 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:49.845698 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-e11735ed-a62c-49d9-a5dc-afac102a13a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:49.983543 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bda8ed7-05b3-4db8-941b-a8bda524bb6d None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:49.991323 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 217/429] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:50.023654 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11735ed-a62c-49d9-a5dc-afac102a13a6 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:50.028314 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:50.035816 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f735cdcc-ada6-4fc4-850a-11d5df352d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:50.036813 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f735cdcc-ada6-4fc4-850a-11d5df352d07 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:50.037283 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f735cdcc-ada6-4fc4-850a-11d5df352d07 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:50.037753 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f735cdcc-ada6-4fc4-850a-11d5df352d07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:50.099538 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:50.137434 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f735cdcc-ada6-4fc4-850a-11d5df352d07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:50.139709 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f735cdcc-ada6-4fc4-850a-11d5df352d07 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:50.175131 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11735ed-a62c-49d9-a5dc-afac102a13a6 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:50.218347 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21bb326-b823-440f-a4cf-da3af9865dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:50.219718 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bb326-b823-440f-a4cf-da3af9865dca None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:50.220435 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bb326-b823-440f-a4cf-da3af9865dca None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:50.221365 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bb326-b823-440f-a4cf-da3af9865dca None admin] PATH_INFO: `/v3/roles/member` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:50.237114 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21bb326-b823-440f-a4cf-da3af9865dca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:50.244800 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 213/430] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:43:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 01:43:50.248060 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-b21bb326-b823-440f-a4cf-da3af9865dca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 01:43:50.248060 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:50.248060 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:50.248060 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:50.248060 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:50.248060 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:50.248060 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:50.248060 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:50.248060 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:50.248060 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:50.248060 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:50.248060 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:50.248060 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:50.248060 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:50.248060 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:50.248060 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:50.249098 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:50.249098 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:50.249098 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:50.249098 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:50.249098 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:50.249098 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:50.249098 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:50.249098 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:50.249098 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:50.249098 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:50.249098 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:50.249098 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:50.249098 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:50.249098 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:50.249098 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:50.249098 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:50.250112 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:50.250112 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:50.250112 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:50.250112 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:50.250112 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:50.250112 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:50.250112 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:50.250112 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:50.250112 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:50.250112 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:50.250112 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 01:43:50.250112 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:43:50.251552 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f735cdcc-ada6-4fc4-850a-11d5df352d07 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:50.252560 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 218/431] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:43:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:43:50.262960 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21bb326-b823-440f-a4cf-da3af9865dca None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:50.296031 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:50.382580 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f735cdcc-ada6-4fc4-850a-11d5df352d07 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:50.395840 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21bb326-b823-440f-a4cf-da3af9865dca None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:50.414292 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:50.437806 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d6cf1-25ad-49ce-aa30-ac0e61978937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:50.438937 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d6cf1-25ad-49ce-aa30-ac0e61978937 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:50.439252 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d6cf1-25ad-49ce-aa30-ac0e61978937 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:50.439534 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d6cf1-25ad-49ce-aa30-ac0e61978937 None admin] PATH_INFO: `/v3/projects/e67bbf3bcff7420690a69b58a6241244` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:50.440998 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548d6cf1-25ad-49ce-aa30-ac0e61978937 None admin] RBAC: Authorizing `identity:get_project(project_id=e67bbf3bcff7420690a69b58a6241244)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:50.443261 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548d6cf1-25ad-49ce-aa30-ac0e61978937 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:50.448359 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c3411-f286-4414-bba6-7d125032c649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:50.449232 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c3411-f286-4414-bba6-7d125032c649 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:50.449549 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 214/432] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:43:50 2022] GET /identity/v3/projects/e67bbf3bcff7420690a69b58a6241244 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:50.450138 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c3411-f286-4414-bba6-7d125032c649 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:50.450544 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c3411-f286-4414-bba6-7d125032c649 None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:50.450743 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d6cf1-25ad-49ce-aa30-ac0e61978937 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:50.451739 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37c3411-f286-4414-bba6-7d125032c649 None admin] RBAC: Adding query filter params (name=member) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:50.452205 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37c3411-f286-4414-bba6-7d125032c649 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:50.453973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37c3411-f286-4414-bba6-7d125032c649 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:50.467322 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 219/433] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jun 23 01:43:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:43:50.472461 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c3411-f286-4414-bba6-7d125032c649 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:50.592764 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:50.599671 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d6cf1-25ad-49ce-aa30-ac0e61978937 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:50.608158 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c3411-f286-4414-bba6-7d125032c649 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:50.642384 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d9d0bd-9c78-4d32-a9f7-595e303083a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:50.643515 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9d0bd-9c78-4d32-a9f7-595e303083a2 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:50.643888 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95b8d9f-d748-4fb8-86cf-14e26008b818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:50.644191 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9d0bd-9c78-4d32-a9f7-595e303083a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:50.644606 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9d0bd-9c78-4d32-a9f7-595e303083a2 None admin] PATH_INFO: `/v3/groups/7a6d9470aae44aa9aa3f20c238c5a421` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:50.644912 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b8d9f-d748-4fb8-86cf-14e26008b818 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:50.645282 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b8d9f-d748-4fb8-86cf-14e26008b818 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:50.645674 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b8d9f-d748-4fb8-86cf-14e26008b818 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:50.646502 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d9d0bd-9c78-4d32-a9f7-595e303083a2 None admin] RBAC: Authorizing `identity:get_group(group_id=7a6d9470aae44aa9aa3f20c238c5a421)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:50.648226 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d9d0bd-9c78-4d32-a9f7-595e303083a2 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:50.654944 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 215/434] 10.222.0.26 () {56 vars in 1180 bytes} [Thu Jun 23 01:43:50 2022] GET /identity/v3/groups/7a6d9470aae44aa9aa3f20c238c5a421 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:43:50.656394 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d9d0bd-9c78-4d32-a9f7-595e303083a2 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:50.692257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:50.718179 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95b8d9f-d748-4fb8-86cf-14e26008b818 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:50.719963 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95b8d9f-d748-4fb8-86cf-14e26008b818 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:50.768366 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d9d0bd-9c78-4d32-a9f7-595e303083a2 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:50.809444 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee32da5-d105-42a7-a1d7-b55cb7fa864a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:50.810484 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee32da5-d105-42a7-a1d7-b55cb7fa864a None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:50.811007 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee32da5-d105-42a7-a1d7-b55cb7fa864a None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:50.811528 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee32da5-d105-42a7-a1d7-b55cb7fa864a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:50.812626 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee32da5-d105-42a7-a1d7-b55cb7fa864a None admin] RBAC: Adding query filter params (group.id=7a6d9470aae44aa9aa3f20c238c5a421, role.id=1c7adeca27524e258f7ddd2c107e43e1, scope.project.id=e67bbf3bcff7420690a69b58a6241244) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:50.813137 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee32da5-d105-42a7-a1d7-b55cb7fa864a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:50.815024 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee32da5-d105-42a7-a1d7-b55cb7fa864a None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:50.822306 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 216/435] 10.222.0.26 () {56 vars in 1376 bytes} [Thu Jun 23 01:43:50 2022] GET /identity/v3/role_assignments?group.id=7a6d9470aae44aa9aa3f20c238c5a421&scope.project.id=e67bbf3bcff7420690a69b58a6241244&role.id=1c7adeca27524e258f7ddd2c107e43e1 => generated 611 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:43:50.827795 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 220/436] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:43:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 01:43:50.833225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee32da5-d105-42a7-a1d7-b55cb7fa864a None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:50.867950 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:50.946043 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee32da5-d105-42a7-a1d7-b55cb7fa864a None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:50.989125 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82324b15-cbe7-493a-89c4-b4375a01a2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:50.990004 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82324b15-cbe7-493a-89c4-b4375a01a2e0 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:50.990286 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82324b15-cbe7-493a-89c4-b4375a01a2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:50.990563 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82324b15-cbe7-493a-89c4-b4375a01a2e0 None admin] PATH_INFO: `/v3/projects/9db43d2e143f49efbddf812fd9d03c69` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:50.991961 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82324b15-cbe7-493a-89c4-b4375a01a2e0 None admin] RBAC: Authorizing `identity:get_project(project_id=9db43d2e143f49efbddf812fd9d03c69)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:50.993589 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82324b15-cbe7-493a-89c4-b4375a01a2e0 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:50.996033 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 217/437] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:43:50 2022] GET /identity/v3/projects/9db43d2e143f49efbddf812fd9d03c69 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:50.996553 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e0b14a-8d19-45ea-97df-2e3dc9b7ecaf None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:50.996842 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e0b14a-8d19-45ea-97df-2e3dc9b7ecaf None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:50.997164 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e0b14a-8d19-45ea-97df-2e3dc9b7ecaf None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:50.998668 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 221/438] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 01:43:51.003863 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e329d4-11cf-4240-8c67-3f55ba01519d None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:51.004209 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e329d4-11cf-4240-8c67-3f55ba01519d None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:51.004415 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82324b15-cbe7-493a-89c4-b4375a01a2e0 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:51.004513 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e329d4-11cf-4240-8c67-3f55ba01519d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:51.023075 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-88e329d4-11cf-4240-8c67-3f55ba01519d None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:51.024307 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-88e329d4-11cf-4240-8c67-3f55ba01519d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:51.102987 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 222/439] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:51.125020 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:51.125864 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e329d4-11cf-4240-8c67-3f55ba01519d None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:51.129402 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:51.130293 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82324b15-cbe7-493a-89c4-b4375a01a2e0 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:51.168179 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14734f4d-c0bf-4ea3-ac19-b2771fce341f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:51.169461 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14734f4d-c0bf-4ea3-ac19-b2771fce341f None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:51.169461 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14734f4d-c0bf-4ea3-ac19-b2771fce341f None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:51.169627 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14734f4d-c0bf-4ea3-ac19-b2771fce341f None admin] PATH_INFO: `/v3/groups/7a6d9470aae44aa9aa3f20c238c5a421` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:51.170776 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14734f4d-c0bf-4ea3-ac19-b2771fce341f None admin] RBAC: Authorizing `identity:get_group(group_id=7a6d9470aae44aa9aa3f20c238c5a421)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:51.172832 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14734f4d-c0bf-4ea3-ac19-b2771fce341f None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:51.174854 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 218/440] 10.222.0.26 () {56 vars in 1180 bytes} [Thu Jun 23 01:43:51 2022] GET /identity/v3/groups/7a6d9470aae44aa9aa3f20c238c5a421 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:51.180806 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14734f4d-c0bf-4ea3-ac19-b2771fce341f None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:51.252968 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e329d4-11cf-4240-8c67-3f55ba01519d None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:51.310098 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ec1c6-03c5-4932-8ebe-afd8a25e622a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:51.310098 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ec1c6-03c5-4932-8ebe-afd8a25e622a None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:51.310098 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ec1c6-03c5-4932-8ebe-afd8a25e622a None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:51.310098 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ec1c6-03c5-4932-8ebe-afd8a25e622a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:51.313149 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14734f4d-c0bf-4ea3-ac19-b2771fce341f None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:51.321488 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669ec1c6-03c5-4932-8ebe-afd8a25e622a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:51.329964 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-669ec1c6-03c5-4932-8ebe-afd8a25e622a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 01:43:51.329964 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:51.329964 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:51.329964 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:51.329964 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:51.329964 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:51.329964 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:51.329964 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:51.329964 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:51.329964 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:51.329964 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:51.329964 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:51.329964 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:51.329964 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:51.329964 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:51.329964 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:51.330865 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:51.330865 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:51.330865 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:51.330865 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:51.330865 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:51.330865 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:51.330865 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:51.330865 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:51.330865 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:51.330865 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:51.330865 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:51.330865 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:51.330865 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:51.330865 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:51.330865 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:51.330865 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:51.332038 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:51.332038 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:51.332038 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:51.332038 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:51.332038 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:51.332038 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:51.332038 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:51.332038 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:51.332038 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:51.332038 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:51.332038 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 01:43:51.332038 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:43:51.332038 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 223/441] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Jun 23 01:43:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:43:51.337219 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ec1c6-03c5-4932-8ebe-afd8a25e622a None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:51.354913 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ef34e2-33f1-4a88-b4f2-c2d4140b5919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:51.355629 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef34e2-33f1-4a88-b4f2-c2d4140b5919 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:51.355913 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef34e2-33f1-4a88-b4f2-c2d4140b5919 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:51.356197 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef34e2-33f1-4a88-b4f2-c2d4140b5919 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:51.357041 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ef34e2-33f1-4a88-b4f2-c2d4140b5919 None admin] RBAC: Adding query filter params (group.id=7a6d9470aae44aa9aa3f20c238c5a421, role.id=9021e3c4e3384a309951be4cc1d852e5, scope.project.id=9db43d2e143f49efbddf812fd9d03c69) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:51.357355 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ef34e2-33f1-4a88-b4f2-c2d4140b5919 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:51.358809 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ef34e2-33f1-4a88-b4f2-c2d4140b5919 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:51.365095 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 219/442] 10.222.0.26 () {56 vars in 1376 bytes} [Thu Jun 23 01:43:51 2022] GET /identity/v3/role_assignments?group.id=7a6d9470aae44aa9aa3f20c238c5a421&scope.project.id=9db43d2e143f49efbddf812fd9d03c69&role.id=9021e3c4e3384a309951be4cc1d852e5 => generated 611 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:43:51.409370 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:51.416028 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb52ab-d7da-4b0e-81b4-8ee38bfcb50d None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:51.416295 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb52ab-d7da-4b0e-81b4-8ee38bfcb50d None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:51.416612 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb52ab-d7da-4b0e-81b4-8ee38bfcb50d None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:51.418218 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 220/443] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 01:43:51.424043 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a185fbc0-d1e8-472c-a37a-ed2d43fb9362 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:51.424319 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a185fbc0-d1e8-472c-a37a-ed2d43fb9362 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:51.424613 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a185fbc0-d1e8-472c-a37a-ed2d43fb9362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:51.442789 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-a185fbc0-d1e8-472c-a37a-ed2d43fb9362 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:51.444073 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-a185fbc0-d1e8-472c-a37a-ed2d43fb9362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:51.460293 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ec1c6-03c5-4932-8ebe-afd8a25e622a None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:51.472779 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:51.501918 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a2d76-2ed6-4e4b-a2d0-bed5dec5621f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:51.502604 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a2d76-2ed6-4e4b-a2d0-bed5dec5621f None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:51.502893 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a2d76-2ed6-4e4b-a2d0-bed5dec5621f None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:51.503246 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a2d76-2ed6-4e4b-a2d0-bed5dec5621f None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:51.503955 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3a2d76-2ed6-4e4b-a2d0-bed5dec5621f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:51.504257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3a2d76-2ed6-4e4b-a2d0-bed5dec5621f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:51.505658 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3a2d76-2ed6-4e4b-a2d0-bed5dec5621f None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:51.515112 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 224/444] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:43:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:51.521837 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a2d76-2ed6-4e4b-a2d0-bed5dec5621f None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:51.524533 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 221/445] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 01:43:51.550038 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a185fbc0-d1e8-472c-a37a-ed2d43fb9362 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:51.637804 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a2d76-2ed6-4e4b-a2d0-bed5dec5621f None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:51.668636 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:51.676522 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a185fbc0-d1e8-472c-a37a-ed2d43fb9362 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:51.677943 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd2164-f6ff-4f8c-990c-d611da9ff99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:51.678604 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd2164-f6ff-4f8c-990c-d611da9ff99e None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:51.678881 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd2164-f6ff-4f8c-990c-d611da9ff99e None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:51.679156 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd2164-f6ff-4f8c-990c-d611da9ff99e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:51.718668 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a932932-29a4-4b4e-9124-24b8ec97d9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:51.718668 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a932932-29a4-4b4e-9124-24b8ec97d9cc None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:51.718948 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a932932-29a4-4b4e-9124-24b8ec97d9cc None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:51.719179 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a932932-29a4-4b4e-9124-24b8ec97d9cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:51.731645 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a932932-29a4-4b4e-9124-24b8ec97d9cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:51.739263 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-7a932932-29a4-4b4e-9124-24b8ec97d9cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:43:51.739263 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:51.739263 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:51.739263 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:51.739263 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:51.739263 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:51.739263 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:51.739263 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:51.739263 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:51.739263 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:51.739263 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:51.739263 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:51.739263 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:51.739263 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:51.739263 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:51.739263 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:51.740162 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:51.740162 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:51.740162 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:51.740162 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:51.740162 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:51.740162 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:51.740162 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:51.740162 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:51.740162 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:51.740162 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:51.740162 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:51.740162 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:51.740162 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:51.740162 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:51.740162 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:51.740162 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:51.740948 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:51.740948 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:51.740948 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:51.740948 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:51.740948 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:51.740948 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:51.740948 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:51.740948 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:51.740948 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:51.740948 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:51.740948 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:43:51.740948 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:43:51.740948 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 222/446] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jun 23 01:43:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:43:51.746042 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a932932-29a4-4b4e-9124-24b8ec97d9cc None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:51.754246 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fd2164-f6ff-4f8c-990c-d611da9ff99e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:51.755647 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fd2164-f6ff-4f8c-990c-d611da9ff99e None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:51.769669 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:51.845216 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a932932-29a4-4b4e-9124-24b8ec97d9cc None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:51.852013 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 225/447] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:43:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 01:43:51.857546 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd2164-f6ff-4f8c-990c-d611da9ff99e None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:51.880886 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db9684e-3cc0-4417-85ef-2046f142cb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:51.881545 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9684e-3cc0-4417-85ef-2046f142cb1e None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:51.881818 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9684e-3cc0-4417-85ef-2046f142cb1e None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:51.882078 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9684e-3cc0-4417-85ef-2046f142cb1e None admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:51.882806 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db9684e-3cc0-4417-85ef-2046f142cb1e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:51.883104 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db9684e-3cc0-4417-85ef-2046f142cb1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:51.884555 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db9684e-3cc0-4417-85ef-2046f142cb1e None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:51.884686 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:51.892846 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 223/448] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:43:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:43:51.898428 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db9684e-3cc0-4417-85ef-2046f142cb1e None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:51.956821 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd2164-f6ff-4f8c-990c-d611da9ff99e None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:51.981649 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:51.991999 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471746f7-68ba-4dd5-87ad-398213fe15e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:51.993019 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471746f7-68ba-4dd5-87ad-398213fe15e7 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:51.993347 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471746f7-68ba-4dd5-87ad-398213fe15e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:51.993672 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471746f7-68ba-4dd5-87ad-398213fe15e7 None admin] PATH_INFO: `/v3/projects/9db43d2e143f49efbddf812fd9d03c69` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:51.995004 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471746f7-68ba-4dd5-87ad-398213fe15e7 None admin] RBAC: Authorizing `identity:get_project(project_id=9db43d2e143f49efbddf812fd9d03c69)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:51.996673 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471746f7-68ba-4dd5-87ad-398213fe15e7 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:51.998366 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 226/449] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:43:51 2022] GET /identity/v3/projects/9db43d2e143f49efbddf812fd9d03c69 => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:51.998811 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db9684e-3cc0-4417-85ef-2046f142cb1e None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:52.003803 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471746f7-68ba-4dd5-87ad-398213fe15e7 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:52.035171 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4813388e-0466-4a33-9acd-15f9c4cbe346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:52.035841 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813388e-0466-4a33-9acd-15f9c4cbe346 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:52.036124 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813388e-0466-4a33-9acd-15f9c4cbe346 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:52.036402 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813388e-0466-4a33-9acd-15f9c4cbe346 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:52.101874 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:52.104098 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4813388e-0466-4a33-9acd-15f9c4cbe346 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:52.105577 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4813388e-0466-4a33-9acd-15f9c4cbe346 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:52.108804 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471746f7-68ba-4dd5-87ad-398213fe15e7 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:52.144194 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e70dac-c8df-41f4-9653-007628b92bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:52.145139 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e70dac-c8df-41f4-9653-007628b92bdf None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:52.145426 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e70dac-c8df-41f4-9653-007628b92bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:52.145701 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e70dac-c8df-41f4-9653-007628b92bdf None admin] PATH_INFO: `/v3/groups/7a6d9470aae44aa9aa3f20c238c5a421` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:52.146932 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e70dac-c8df-41f4-9653-007628b92bdf None admin] RBAC: Authorizing `identity:get_group(group_id=7a6d9470aae44aa9aa3f20c238c5a421)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:52.148408 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e70dac-c8df-41f4-9653-007628b92bdf None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:52.150045 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 227/450] 10.222.0.26 () {56 vars in 1180 bytes} [Thu Jun 23 01:43:52 2022] GET /identity/v3/groups/7a6d9470aae44aa9aa3f20c238c5a421 => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:52.155813 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e70dac-c8df-41f4-9653-007628b92bdf None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:52.187501 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:52.203882 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 224/451] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:43:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 01:43:52.209764 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4813388e-0466-4a33-9acd-15f9c4cbe346 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:52.253859 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e70dac-c8df-41f4-9653-007628b92bdf None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:52.290157 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f679e11-d467-4a5a-948f-bff4c225c4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:52.290732 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f679e11-d467-4a5a-948f-bff4c225c4c8 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:52.291009 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f679e11-d467-4a5a-948f-bff4c225c4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:52.291292 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f679e11-d467-4a5a-948f-bff4c225c4c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:52.292112 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f679e11-d467-4a5a-948f-bff4c225c4c8 None admin] RBAC: Adding query filter params (group.id=7a6d9470aae44aa9aa3f20c238c5a421, role.id=1c7adeca27524e258f7ddd2c107e43e1, scope.project.id=9db43d2e143f49efbddf812fd9d03c69) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:52.292416 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f679e11-d467-4a5a-948f-bff4c225c4c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:52.293942 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f679e11-d467-4a5a-948f-bff4c225c4c8 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:52.300101 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 228/452] 10.222.0.26 () {56 vars in 1376 bytes} [Thu Jun 23 01:43:52 2022] GET /identity/v3/role_assignments?group.id=7a6d9470aae44aa9aa3f20c238c5a421&scope.project.id=9db43d2e143f49efbddf812fd9d03c69&role.id=1c7adeca27524e258f7ddd2c107e43e1 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:52.311472 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4813388e-0466-4a33-9acd-15f9c4cbe346 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:52.349594 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3280d308-adac-4c7b-9a12-b53e7e65a153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:52.350470 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3280d308-adac-4c7b-9a12-b53e7e65a153 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:52.350750 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3280d308-adac-4c7b-9a12-b53e7e65a153 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:52.351026 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3280d308-adac-4c7b-9a12-b53e7e65a153 None admin] PATH_INFO: `/v3/projects/cc3d5c56f68947b3b757a0c2a92187ac` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:52.352503 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3280d308-adac-4c7b-9a12-b53e7e65a153 None admin] RBAC: Authorizing `identity:get_project(project_id=cc3d5c56f68947b3b757a0c2a92187ac)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:52.354010 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3280d308-adac-4c7b-9a12-b53e7e65a153 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:52.355826 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 225/453] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:43:52 2022] GET /identity/v3/projects/cc3d5c56f68947b3b757a0c2a92187ac => generated 344 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:52.361690 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f679e11-d467-4a5a-948f-bff4c225c4c8 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:52.392637 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:52.471111 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f679e11-d467-4a5a-948f-bff4c225c4c8 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:52.507659 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e884c7-306a-49b3-8d8c-19e89362d78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:52.508526 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e884c7-306a-49b3-8d8c-19e89362d78f None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:52.508803 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e884c7-306a-49b3-8d8c-19e89362d78f None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:52.509077 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e884c7-306a-49b3-8d8c-19e89362d78f None admin] PATH_INFO: `/v3/groups/f0e4bbd719ad4ad9baf7562c7fccc891` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:52.510374 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e884c7-306a-49b3-8d8c-19e89362d78f None admin] RBAC: Authorizing `identity:get_group(group_id=f0e4bbd719ad4ad9baf7562c7fccc891)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:52.511959 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e884c7-306a-49b3-8d8c-19e89362d78f None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:52.513603 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 229/454] 10.222.0.26 () {56 vars in 1180 bytes} [Thu Jun 23 01:43:52 2022] GET /identity/v3/groups/f0e4bbd719ad4ad9baf7562c7fccc891 => generated 229 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:52.519812 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3280d308-adac-4c7b-9a12-b53e7e65a153 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:52.575290 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:52.625478 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3280d308-adac-4c7b-9a12-b53e7e65a153 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:52.657935 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9da960-7654-49a4-b741-7360b9a6597e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:52.658583 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9da960-7654-49a4-b741-7360b9a6597e None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:52.658864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9da960-7654-49a4-b741-7360b9a6597e None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:52.659139 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9da960-7654-49a4-b741-7360b9a6597e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:52.660018 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9da960-7654-49a4-b741-7360b9a6597e None admin] RBAC: Adding query filter params (group.id=f0e4bbd719ad4ad9baf7562c7fccc891, role.id=7b20d65f49ee4a48894b6c060bb5d580, scope.project.id=cc3d5c56f68947b3b757a0c2a92187ac) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:52.660328 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9da960-7654-49a4-b741-7360b9a6597e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:52.661862 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9da960-7654-49a4-b741-7360b9a6597e None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:52.667564 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 226/455] 10.222.0.26 () {56 vars in 1376 bytes} [Thu Jun 23 01:43:52 2022] GET /identity/v3/role_assignments?group.id=f0e4bbd719ad4ad9baf7562c7fccc891&scope.project.id=cc3d5c56f68947b3b757a0c2a92187ac&role.id=7b20d65f49ee4a48894b6c060bb5d580 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:55.146911 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417a09f1-5355-4674-bbfc-c25af1a46441 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:55.147969 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417a09f1-5355-4674-bbfc-c25af1a46441 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:55.148450 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417a09f1-5355-4674-bbfc-c25af1a46441 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:55.150310 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 230/456] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:55.155353 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b583f7-6f49-4cc5-8049-426beceb8a4b None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:55.156141 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b583f7-6f49-4cc5-8049-426beceb8a4b None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:55.156607 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b583f7-6f49-4cc5-8049-426beceb8a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:55.177247 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-03b583f7-6f49-4cc5-8049-426beceb8a4b None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:55.178464 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-03b583f7-6f49-4cc5-8049-426beceb8a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:55.272684 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 227/457] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:55.305084 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417a09f1-5355-4674-bbfc-c25af1a46441 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:55.446700 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:55.454175 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417a09f1-5355-4674-bbfc-c25af1a46441 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:55.481497 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c3fb7-6aae-4a04-a4af-dd39dd3a8ed0 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:55.482113 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c3fb7-6aae-4a04-a4af-dd39dd3a8ed0 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:55.482583 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c3fb7-6aae-4a04-a4af-dd39dd3a8ed0 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:55.484444 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 228/458] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:55.489627 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c2964-3437-4ef1-b7dd-a9a7db01a1e5 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:55.490142 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c2964-3437-4ef1-b7dd-a9a7db01a1e5 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:55.490653 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c2964-3437-4ef1-b7dd-a9a7db01a1e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:55.510541 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a076e3-5b2d-445c-a8b1-9c0dcb8f36a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:55.511445 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a076e3-5b2d-445c-a8b1-9c0dcb8f36a8 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:55.511724 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a076e3-5b2d-445c-a8b1-9c0dcb8f36a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:55.512000 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a076e3-5b2d-445c-a8b1-9c0dcb8f36a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:55.525746 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-114c2964-3437-4ef1-b7dd-a9a7db01a1e5 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:55.526955 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-114c2964-3437-4ef1-b7dd-a9a7db01a1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:55.533608 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a076e3-5b2d-445c-a8b1-9c0dcb8f36a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:55.535107 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a076e3-5b2d-445c-a8b1-9c0dcb8f36a8 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:55.558409 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-b9a076e3-5b2d-445c-a8b1-9c0dcb8f36a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:43:55.558409 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:55.558409 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:55.558409 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:55.558409 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:55.558409 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:55.558409 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:55.558409 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:55.558409 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:55.558409 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:55.558409 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:55.558409 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:55.558409 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:43:55.558409 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:43:55.558409 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:43:55.559537 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:43:55.559537 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:55.559537 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:55.559537 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:55.559537 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:55.559537 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:55.559537 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:55.559537 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:55.559537 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:55.559537 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:55.559537 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:55.559537 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:55.559537 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:55.559537 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:55.559537 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:55.559537 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:55.559537 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:55.560682 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:43:55.560682 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:43:55.560682 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:43:55.560682 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:43:55.562548 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 231/459] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:43:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:43:55.564061 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96130788-12ab-4308-9eb8-8386292e6ec3 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:55.564466 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96130788-12ab-4308-9eb8-8386292e6ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:55.564876 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96130788-12ab-4308-9eb8-8386292e6ec3 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:55.570307 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 232/460] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:55.571078 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96130788-12ab-4308-9eb8-8386292e6ec3 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:55.589738 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:55.624692 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 229/461] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 01:43:55.626752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ba70e-35e7-4f2c-9563-b2b15db19f95 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:55.627039 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ba70e-35e7-4f2c-9563-b2b15db19f95 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:55.627336 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ba70e-35e7-4f2c-9563-b2b15db19f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:55.644110 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-c00ba70e-35e7-4f2c-9563-b2b15db19f95 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:55.645400 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-c00ba70e-35e7-4f2c-9563-b2b15db19f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:55.696185 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96130788-12ab-4308-9eb8-8386292e6ec3 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:55.727413 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 230/462] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:55.728267 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00ba70e-35e7-4f2c-9563-b2b15db19f95 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:55.737534 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b8b9e2-091f-4f76-9014-b411bb82da77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:55.738942 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b8b9e2-091f-4f76-9014-b411bb82da77 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:55.739469 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b8b9e2-091f-4f76-9014-b411bb82da77 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:55.739941 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b8b9e2-091f-4f76-9014-b411bb82da77 None admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:55.740892 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b8b9e2-091f-4f76-9014-b411bb82da77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:55.741392 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b8b9e2-091f-4f76-9014-b411bb82da77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:55.742960 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b8b9e2-091f-4f76-9014-b411bb82da77 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:55.764915 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 233/463] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:43:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:55.769720 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b8b9e2-091f-4f76-9014-b411bb82da77 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:55.783037 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:55.852378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00ba70e-35e7-4f2c-9563-b2b15db19f95 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:55.860198 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:55.877167 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b8b9e2-091f-4f76-9014-b411bb82da77 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:55.892377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25a693-d0e9-4a5c-8989-3b02300e853d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:55.893223 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25a693-d0e9-4a5c-8989-3b02300e853d None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:55.893511 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25a693-d0e9-4a5c-8989-3b02300e853d None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:55.893782 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25a693-d0e9-4a5c-8989-3b02300e853d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:55.904403 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df25a693-d0e9-4a5c-8989-3b02300e853d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:55.905887 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df25a693-d0e9-4a5c-8989-3b02300e853d None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:55.913337 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b12bfe-27b3-4f21-94f7-6f946a239128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:55.914144 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b12bfe-27b3-4f21-94f7-6f946a239128 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:55.914419 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b12bfe-27b3-4f21-94f7-6f946a239128 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:55.914701 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b12bfe-27b3-4f21-94f7-6f946a239128 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:55.924613 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-df25a693-d0e9-4a5c-8989-3b02300e853d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:43:55.924613 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:55.924613 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:55.924613 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:55.924613 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:55.924613 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:55.924613 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:55.924613 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:55.924613 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:55.924613 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:55.924613 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:55.924613 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:55.924613 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:43:55.924613 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:43:55.924613 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:43:55.925611 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:43:55.925611 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:55.925611 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:55.925611 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:55.925611 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:55.925611 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:55.925611 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:55.925611 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:55.925611 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:55.925611 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:55.925611 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:55.925611 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:55.925611 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:55.925611 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:55.925611 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:55.925611 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:55.925611 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:55.926749 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:43:55.926749 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:43:55.926749 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:43:55.926749 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:43:55.927730 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 231/464] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:43:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:43:55.928135 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b12bfe-27b3-4f21-94f7-6f946a239128 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:55.928445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25a693-d0e9-4a5c-8989-3b02300e853d None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:55.929724 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b12bfe-27b3-4f21-94f7-6f946a239128 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:55.948077 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-42b12bfe-27b3-4f21-94f7-6f946a239128 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:43:55.948077 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:55.948077 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:55.948077 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:55.948077 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:55.948077 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:55.948077 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:55.948077 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:55.948077 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:55.948077 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:55.948077 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:55.948077 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:55.948077 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:43:55.948077 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:43:55.948077 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:43:55.948978 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:43:55.948978 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:55.948978 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:55.948978 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:55.948978 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:55.948978 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:55.948978 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:55.948978 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:55.948978 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:55.948978 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:55.948978 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:55.948978 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:55.948978 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:55.948978 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:55.948978 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:55.948978 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:55.948978 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:55.949800 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:43:55.949800 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:43:55.949800 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:43:55.949800 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:43:55.949800 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 234/465] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:43:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:43:55.952995 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1136a64e-3ff1-4fce-9e2c-a90d14bb699e None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:55.953494 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1136a64e-3ff1-4fce-9e2c-a90d14bb699e None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:55.953972 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1136a64e-3ff1-4fce-9e2c-a90d14bb699e None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:55.959470 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 235/466] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:55 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:55.960180 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1136a64e-3ff1-4fce-9e2c-a90d14bb699e None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:56.023446 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:56.049484 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25a693-d0e9-4a5c-8989-3b02300e853d None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:56.070278 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1136a64e-3ff1-4fce-9e2c-a90d14bb699e None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:56.087677 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f5e94-4069-4745-a711-406426ac5ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:56.088321 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f5e94-4069-4745-a711-406426ac5ac9 None admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:56.088634 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f5e94-4069-4745-a711-406426ac5ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:56.088905 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f5e94-4069-4745-a711-406426ac5ac9 None admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:56.089609 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8f5e94-4069-4745-a711-406426ac5ac9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:56.091049 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8f5e94-4069-4745-a711-406426ac5ac9 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:56.106407 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0227be67-c9a1-40cd-b377-fac83798914f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:56.107141 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227be67-c9a1-40cd-b377-fac83798914f None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:56.107536 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227be67-c9a1-40cd-b377-fac83798914f None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:56.107963 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227be67-c9a1-40cd-b377-fac83798914f None admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:56.108687 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0227be67-c9a1-40cd-b377-fac83798914f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:56.108991 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0227be67-c9a1-40cd-b377-fac83798914f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:56.110366 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0227be67-c9a1-40cd-b377-fac83798914f None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:56.113549 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 232/467] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Jun 23 01:43:55 2022] POST /identity/v3/users => generated 246 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:43:56.114547 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f5e94-4069-4745-a711-406426ac5ac9 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:56.135004 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 236/468] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:43:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:56.135004 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf29b3d0-7ad7-4424-84e5-ab77187c6bf0 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:56.135004 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf29b3d0-7ad7-4424-84e5-ab77187c6bf0 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:56.135004 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf29b3d0-7ad7-4424-84e5-ab77187c6bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:56.140497 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:56.154701 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-cf29b3d0-7ad7-4424-84e5-ab77187c6bf0 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:56.156643 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-cf29b3d0-7ad7-4424-84e5-ab77187c6bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:56.235084 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 237/469] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:56.236315 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf29b3d0-7ad7-4424-84e5-ab77187c6bf0 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:56.265119 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f5e94-4069-4745-a711-406426ac5ac9 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:56.300320 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:56.304318 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5304693-f672-4aec-8103-899ee9e73019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:56.304989 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5304693-f672-4aec-8103-899ee9e73019 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:56.305626 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5304693-f672-4aec-8103-899ee9e73019 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:56.305901 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5304693-f672-4aec-8103-899ee9e73019 None admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:56.306640 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5304693-f672-4aec-8103-899ee9e73019 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:56.306945 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5304693-f672-4aec-8103-899ee9e73019 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:56.308793 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5304693-f672-4aec-8103-899ee9e73019 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:56.322780 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 233/470] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:43:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:56.325593 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5304693-f672-4aec-8103-899ee9e73019 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:56.372164 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf29b3d0-7ad7-4424-84e5-ab77187c6bf0 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:56.408387 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b77716f-5e13-4f4e-97c8-b0ae6e75d710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:56.409073 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b77716f-5e13-4f4e-97c8-b0ae6e75d710 None admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:56.409355 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b77716f-5e13-4f4e-97c8-b0ae6e75d710 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:56.409632 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b77716f-5e13-4f4e-97c8-b0ae6e75d710 None admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:56.410402 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b77716f-5e13-4f4e-97c8-b0ae6e75d710 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:56.411901 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b77716f-5e13-4f4e-97c8-b0ae6e75d710 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:56.413221 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:56.432542 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 238/471] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Jun 23 01:43:56 2022] POST /identity/v3/users => generated 248 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:43:56.433159 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b77716f-5e13-4f4e-97c8-b0ae6e75d710 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:56.439062 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5304693-f672-4aec-8103-899ee9e73019 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:56.483951 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5cefdd-e701-449d-9364-e45deb15f6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:56.484801 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5cefdd-e701-449d-9364-e45deb15f6ec None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:56.485123 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5cefdd-e701-449d-9364-e45deb15f6ec None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:56.485438 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5cefdd-e701-449d-9364-e45deb15f6ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:56.498169 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5cefdd-e701-449d-9364-e45deb15f6ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:56.499802 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5cefdd-e701-449d-9364-e45deb15f6ec None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:56.520629 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-7a5cefdd-e701-449d-9364-e45deb15f6ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:43:56.520629 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:56.520629 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:56.520629 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:56.520629 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:56.520629 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:56.520629 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:56.520629 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:56.520629 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:56.520629 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:56.520629 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:56.520629 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:56.520629 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:43:56.520629 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:43:56.520629 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:43:56.521554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:43:56.521554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:56.521554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:56.521554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:56.521554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:56.521554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:56.521554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:56.521554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:56.521554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:56.521554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:56.521554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:56.521554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:56.521554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:56.521554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:56.521554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:56.521554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:56.521554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:56.522498 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:43:56.522498 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:43:56.522498 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:43:56.522498 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:43:56.522498 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 234/472] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:43:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:43:56.523497 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331de1d-1cc3-4429-9d89-b22eeed7a4a9 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:56.523597 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331de1d-1cc3-4429-9d89-b22eeed7a4a9 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:56.523863 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331de1d-1cc3-4429-9d89-b22eeed7a4a9 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:56.529511 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 235/473] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:56.530619 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6331de1d-1cc3-4429-9d89-b22eeed7a4a9 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:56.563684 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b77716f-5e13-4f4e-97c8-b0ae6e75d710 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:56.567106 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:56.605491 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4dc8ab-605a-482b-ac0e-75efa7ef1bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:56.606227 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4dc8ab-605a-482b-ac0e-75efa7ef1bb7 None admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:56.606510 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4dc8ab-605a-482b-ac0e-75efa7ef1bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:56.607498 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4dc8ab-605a-482b-ac0e-75efa7ef1bb7 None admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:56.608533 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4dc8ab-605a-482b-ac0e-75efa7ef1bb7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:56.610000 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4dc8ab-605a-482b-ac0e-75efa7ef1bb7 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:56.634042 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 239/474] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Jun 23 01:43:56 2022] POST /identity/v3/users => generated 248 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:43:56.635126 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c93e0c-7dd7-459f-a458-43dd7f5bb906 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:56.635459 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c93e0c-7dd7-459f-a458-43dd7f5bb906 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:56.635728 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c93e0c-7dd7-459f-a458-43dd7f5bb906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:56.658161 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-f7c93e0c-7dd7-459f-a458-43dd7f5bb906 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:56.658161 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-f7c93e0c-7dd7-459f-a458-43dd7f5bb906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:56.665294 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6331de1d-1cc3-4429-9d89-b22eeed7a4a9 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:56.705717 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:56.719913 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809bfe7a-953a-4dea-a89e-7380478b4e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:56.720607 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809bfe7a-953a-4dea-a89e-7380478b4e82 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:56.720889 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809bfe7a-953a-4dea-a89e-7380478b4e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:56.721162 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809bfe7a-953a-4dea-a89e-7380478b4e82 None admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:56.722413 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809bfe7a-953a-4dea-a89e-7380478b4e82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:56.722413 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809bfe7a-953a-4dea-a89e-7380478b4e82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:56.723822 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809bfe7a-953a-4dea-a89e-7380478b4e82 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:56.737821 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 236/475] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:43:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:56.743512 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809bfe7a-953a-4dea-a89e-7380478b4e82 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:56.758826 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 240/476] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:56.802692 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c93e0c-7dd7-459f-a458-43dd7f5bb906 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:56.870215 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809bfe7a-953a-4dea-a89e-7380478b4e82 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:56.913448 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef2d5f-41c8-461e-8c56-f5ef8ae08309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:56.914288 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef2d5f-41c8-461e-8c56-f5ef8ae08309 None admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:56.914670 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef2d5f-41c8-461e-8c56-f5ef8ae08309 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:56.915042 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef2d5f-41c8-461e-8c56-f5ef8ae08309 None admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:56.915939 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eef2d5f-41c8-461e-8c56-f5ef8ae08309 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:56.917612 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eef2d5f-41c8-461e-8c56-f5ef8ae08309 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:56.927427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:56.943010 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c93e0c-7dd7-459f-a458-43dd7f5bb906 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:56.945684 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 237/477] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Jun 23 01:43:56 2022] POST /identity/v3/users => generated 249 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:43:57.004780 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b490f328-115c-446d-83b9-e956959d1b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:57.005486 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490f328-115c-446d-83b9-e956959d1b46 None admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:57.005811 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490f328-115c-446d-83b9-e956959d1b46 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:57.006091 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490f328-115c-446d-83b9-e956959d1b46 None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:57.006868 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b490f328-115c-446d-83b9-e956959d1b46 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:57.008374 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b490f328-115c-446d-83b9-e956959d1b46 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:57.018961 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.sql.core [None req-b490f328-115c-446d-83b9-e956959d1b46 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 23 01:43:57.018961 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 23 01:43:57.018961 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [parameters: {'id': '778b44fdd2174cc5a99cdbed3ffc1065', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 23 01:43:57.018961 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80459) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 23 01:43:57.036639 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-b490f328-115c-446d-83b9-e956959d1b46 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 23 01:43:57.036639 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:57.036639 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 01:43:57.036639 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 01:43:57.036639 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 01:43:57.036639 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 01:43:57.036639 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 01:43:57.036639 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 01:43:57.036639 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 01:43:57.036639 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application conn.query(q) Jun 23 01:43:57.036639 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 01:43:57.036639 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 01:43:57.036639 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 01:43:57.036639 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application result.read() Jun 23 01:43:57.036639 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 01:43:57.037806 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 01:43:57.037806 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 01:43:57.037806 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 01:43:57.037806 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 01:43:57.037806 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 01:43:57.037806 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 01:43:57.037806 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 01:43:57.037806 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 23 01:43:57.037806 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:43:57.037806 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 23 01:43:57.037806 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:43:57.037806 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:57.037806 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 23 01:43:57.037806 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 23 01:43:57.037806 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 23 01:43:57.037806 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return ref.to_dict() Jun 23 01:43:57.037806 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 01:43:57.037806 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application next(self.gen) Jun 23 01:43:57.039963 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 23 01:43:57.039963 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application yield resource Jun 23 01:43:57.039963 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 01:43:57.039963 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application next(self.gen) Jun 23 01:43:57.039963 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 23 01:43:57.039963 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application self.session.rollback() Jun 23 01:43:57.039963 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 23 01:43:57.039963 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application self.force_reraise() Jun 23 01:43:57.039963 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 23 01:43:57.039963 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise self.value Jun 23 01:43:57.039963 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 23 01:43:57.039963 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 23 01:43:57.039963 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 23 01:43:57.039963 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application session.commit() Jun 23 01:43:57.039963 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 23 01:43:57.039963 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 23 01:43:57.039963 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 23 01:43:57.041913 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application self._prepare_impl() Jun 23 01:43:57.041913 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 23 01:43:57.041913 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application self.session.flush() Jun 23 01:43:57.041913 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 23 01:43:57.041913 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application self._flush(objects) Jun 23 01:43:57.041913 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 23 01:43:57.041913 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 23 01:43:57.041913 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 23 01:43:57.041913 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application compat.raise_( Jun 23 01:43:57.041913 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 01:43:57.041913 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception Jun 23 01:43:57.041913 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 23 01:43:57.041913 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application flush_context.execute() Jun 23 01:43:57.041913 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 23 01:43:57.041913 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rec.execute(self) Jun 23 01:43:57.041913 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 23 01:43:57.041913 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 23 01:43:57.043301 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 23 01:43:57.043301 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 23 01:43:57.043301 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 23 01:43:57.043301 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 23 01:43:57.043301 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 23 01:43:57.043301 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 23 01:43:57.043301 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 23 01:43:57.043301 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 23 01:43:57.043301 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 23 01:43:57.043301 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 23 01:43:57.043301 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 23 01:43:57.043301 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 23 01:43:57.043301 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 23 01:43:57.043301 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 23 01:43:57.043301 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 01:43:57.043301 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception Jun 23 01:43:57.045328 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 01:43:57.045328 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 01:43:57.045328 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 01:43:57.045328 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 01:43:57.045328 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 01:43:57.045328 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 01:43:57.045328 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 01:43:57.045328 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application conn.query(q) Jun 23 01:43:57.045328 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 01:43:57.045328 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 01:43:57.045328 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 01:43:57.045328 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application result.read() Jun 23 01:43:57.045328 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 01:43:57.045328 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 01:43:57.045328 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 01:43:57.045328 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 01:43:57.047440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 01:43:57.047440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 01:43:57.047440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 01:43:57.047440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 01:43:57.047440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 23 01:43:57.047440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 23 01:43:57.047440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [parameters: {'id': '778b44fdd2174cc5a99cdbed3ffc1065', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 23 01:43:57.047440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 23 01:43:57.047440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:43:57.047440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 01:43:57.047440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:43:57.047440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:57.047440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:57.047440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:57.047440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:57.047440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:57.055628 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:57.055628 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:57.055628 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:57.055628 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:57.055628 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:57.055628 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:57.055628 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 23 01:43:57.055628 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 23 01:43:57.055628 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:57.055628 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:57.055628 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 23 01:43:57.055628 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 23 01:43:57.055628 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 23 01:43:57.055628 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 23 01:43:57.055628 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 23 01:43:57.055628 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:43:57.057560 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 241/478] 10.222.0.26 () {60 vars in 1131 bytes} [Thu Jun 23 01:43:56 2022] POST /identity/v3/roles => generated 144 bytes in 236 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:43:57.057682 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef2d5f-41c8-461e-8c56-f5ef8ae08309 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:57.104341 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:57.166960 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef2d5f-41c8-461e-8c56-f5ef8ae08309 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:57.201670 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0fc62b-2e3c-4823-9915-35f982486263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:57.202487 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0fc62b-2e3c-4823-9915-35f982486263 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:57.202770 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0fc62b-2e3c-4823-9915-35f982486263 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:57.203042 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0fc62b-2e3c-4823-9915-35f982486263 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:57.214099 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0fc62b-2e3c-4823-9915-35f982486263 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:57.222338 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-eb0fc62b-2e3c-4823-9915-35f982486263 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 01:43:57.222338 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:57.222338 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:57.222338 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:57.222338 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:57.222338 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:57.222338 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:57.222338 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:57.222338 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:57.222338 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:57.222338 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:57.222338 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:57.222338 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:57.222338 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:57.222338 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:57.222338 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:57.223520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:57.223520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:57.223520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:57.223520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:57.223520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:57.223520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:57.223520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:57.223520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:57.223520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:57.223520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:57.223520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:57.223520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:57.223520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:57.223520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:57.223520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:57.223520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:57.224606 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:57.224606 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:57.224606 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:57.224606 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:57.224606 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:57.224606 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:57.224606 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:57.224606 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:57.224606 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:57.224606 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:57.224606 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 01:43:57.224606 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:43:57.224606 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 238/479] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Jun 23 01:43:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:43:57.229401 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b490f328-115c-446d-83b9-e956959d1b46 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:57.354537 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b490f328-115c-446d-83b9-e956959d1b46 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:57.389962 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f2782-5ece-4a9b-943b-042166356b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:57.390642 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f2782-5ece-4a9b-943b-042166356b68 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:57.390924 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f2782-5ece-4a9b-943b-042166356b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:57.391198 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f2782-5ece-4a9b-943b-042166356b68 None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:57.391978 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f2782-5ece-4a9b-943b-042166356b68 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:57.392283 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f2782-5ece-4a9b-943b-042166356b68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:57.393667 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f2782-5ece-4a9b-943b-042166356b68 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:57.402786 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 242/480] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:43:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:58.636726 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d94dd-4382-4950-9d1e-300ffb499eb1 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:58.637236 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d94dd-4382-4950-9d1e-300ffb499eb1 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:58.637236 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d94dd-4382-4950-9d1e-300ffb499eb1 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:58.638845 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 239/481] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:58.643674 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5988b-e77f-4cc3-a752-61a5592f4ccb None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:58.644053 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5988b-e77f-4cc3-a752-61a5592f4ccb None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:58.644227 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5988b-e77f-4cc3-a752-61a5592f4ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:58.661346 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-63f5988b-e77f-4cc3-a752-61a5592f4ccb None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:58.662542 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-63f5988b-e77f-4cc3-a752-61a5592f4ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:58.673750 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:58.749284 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 243/482] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:58.784309 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d94dd-4382-4950-9d1e-300ffb499eb1 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:58.896035 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d94dd-4382-4950-9d1e-300ffb499eb1 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:58.897564 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:58.946617 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbef77b4-28cb-48d6-be6f-9b9cbfaea1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:58.947444 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef77b4-28cb-48d6-be6f-9b9cbfaea1be None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:58.947730 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef77b4-28cb-48d6-be6f-9b9cbfaea1be None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:58.948008 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef77b4-28cb-48d6-be6f-9b9cbfaea1be None admin] PATH_INFO: `/v3/roles/service` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:58.959032 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbef77b4-28cb-48d6-be6f-9b9cbfaea1be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:58.970882 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-fbef77b4-28cb-48d6-be6f-9b9cbfaea1be None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 01:43:58.970882 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:58.970882 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:58.970882 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:58.970882 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:58.970882 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:58.970882 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:58.970882 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:58.970882 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:58.970882 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:58.970882 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:58.970882 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:58.970882 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:58.970882 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:58.970882 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:58.970882 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:58.971798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:58.971798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:58.971798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:58.971798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:58.971798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:58.971798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:58.971798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:58.971798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:58.971798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:58.971798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:58.971798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:58.971798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:58.971798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:58.971798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:58.971798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:58.971798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:58.972568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:58.972568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:58.972568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:58.972568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:58.972568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:58.972568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:58.972568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:58.972568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:58.972568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:58.972568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:58.972568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 01:43:58.972568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:43:58.972568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 240/483] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jun 23 01:43:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:43:58.977931 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f5988b-e77f-4cc3-a752-61a5592f4ccb None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:59.078713 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc34b022-c05a-4462-9920-99937bd691a9 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:59.079033 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc34b022-c05a-4462-9920-99937bd691a9 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:59.079359 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc34b022-c05a-4462-9920-99937bd691a9 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:59.080741 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 241/484] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:59.086674 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ecacc-1fb6-4e50-b9b5-63e7b0792e07 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:59.087448 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ecacc-1fb6-4e50-b9b5-63e7b0792e07 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:59.087941 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ecacc-1fb6-4e50-b9b5-63e7b0792e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:59.114042 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f5988b-e77f-4cc3-a752-61a5592f4ccb None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:59.126361 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-fa1ecacc-1fb6-4e50-b9b5-63e7b0792e07 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:59.127615 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-fa1ecacc-1fb6-4e50-b9b5-63e7b0792e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:59.184323 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39fb5c-0583-4f2c-b047-7ce6cc18bf51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:59.185020 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39fb5c-0583-4f2c-b047-7ce6cc18bf51 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:59.185298 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39fb5c-0583-4f2c-b047-7ce6cc18bf51 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:59.185568 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39fb5c-0583-4f2c-b047-7ce6cc18bf51 None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:59.186328 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f39fb5c-0583-4f2c-b047-7ce6cc18bf51 None admin] RBAC: Adding query filter params (name=service) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:59.186638 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f39fb5c-0583-4f2c-b047-7ce6cc18bf51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:59.188057 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f39fb5c-0583-4f2c-b047-7ce6cc18bf51 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:59.196894 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 244/485] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Jun 23 01:43:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:59.208220 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39fb5c-0583-4f2c-b047-7ce6cc18bf51 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:59.225345 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:59.238204 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 242/486] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:59.289367 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1ecacc-1fb6-4e50-b9b5-63e7b0792e07 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:59.373625 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39fb5c-0583-4f2c-b047-7ce6cc18bf51 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:59.412014 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89162f8-cf69-4504-b7db-e312da980c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:59.412808 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89162f8-cf69-4504-b7db-e312da980c98 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:59.413081 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89162f8-cf69-4504-b7db-e312da980c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:59.413347 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89162f8-cf69-4504-b7db-e312da980c98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:59.422050 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1ecacc-1fb6-4e50-b9b5-63e7b0792e07 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:59.424458 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89162f8-cf69-4504-b7db-e312da980c98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:59.424630 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:59.425955 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89162f8-cf69-4504-b7db-e312da980c98 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:59.445178 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-b89162f8-cf69-4504-b7db-e312da980c98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:43:59.445178 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:59.445178 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:59.445178 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:59.445178 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:59.445178 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:59.445178 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:59.445178 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:59.445178 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:59.445178 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:59.445178 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:59.445178 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:59.445178 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:43:59.445178 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:43:59.445178 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:43:59.446247 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:43:59.446247 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:59.446247 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:59.446247 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:59.446247 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:59.446247 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:59.446247 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:59.446247 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:59.446247 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:59.446247 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:59.446247 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:59.446247 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:59.446247 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:59.446247 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:59.446247 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:59.446247 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:59.446247 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:59.447323 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:43:59.447323 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:43:59.447323 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:43:59.447323 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:43:59.447323 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 245/487] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:43:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:43:59.452062 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77a2f5-2f05-4759-8469-425c72d055d4 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:59.452348 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77a2f5-2f05-4759-8469-425c72d055d4 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:59.452636 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77a2f5-2f05-4759-8469-425c72d055d4 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:59.454245 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 246/488] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:59.455583 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf77a2f5-2f05-4759-8469-425c72d055d4 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:59.469768 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec6be9-5f78-44c9-950a-0857984170cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:59.470814 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec6be9-5f78-44c9-950a-0857984170cb None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:59.471092 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec6be9-5f78-44c9-950a-0857984170cb None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:59.471399 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec6be9-5f78-44c9-950a-0857984170cb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:59.486206 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acec6be9-5f78-44c9-950a-0857984170cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:59.496396 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-acec6be9-5f78-44c9-950a-0857984170cb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 01:43:59.496396 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:59.496396 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:59.496396 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:59.496396 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:59.496396 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:59.496396 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:59.496396 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:59.496396 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:59.496396 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:59.496396 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:59.496396 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:59.496396 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:59.496396 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:59.496396 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:59.496396 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:59.497279 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:59.497279 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:59.497279 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:59.497279 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:59.497279 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:59.497279 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:59.497279 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:59.497279 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:59.497279 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:59.497279 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:59.497279 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:59.497279 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:59.497279 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:59.497279 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:59.497279 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:59.497279 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:59.498055 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:59.498055 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:59.498055 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:59.498055 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:59.498055 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:59.498055 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:59.498055 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:59.498055 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:59.498055 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:59.498055 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:59.498055 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 01:43:59.498055 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:43:59.499101 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 243/489] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jun 23 01:43:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:43:59.502442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4ad73-5641-48be-afe2-dbcb254f16b0 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:59.502442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4ad73-5641-48be-afe2-dbcb254f16b0 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:59.502442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4ad73-5641-48be-afe2-dbcb254f16b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:59.518365 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-dda4ad73-5641-48be-afe2-dbcb254f16b0 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:59.519600 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-dda4ad73-5641-48be-afe2-dbcb254f16b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:59.522002 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:59.570119 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf77a2f5-2f05-4759-8469-425c72d055d4 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:59.589574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 244/490] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:59.589574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da01a70-2b8b-4ed6-af87-a757ba4e0565 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:59.589574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da01a70-2b8b-4ed6-af87-a757ba4e0565 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:59.589574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da01a70-2b8b-4ed6-af87-a757ba4e0565 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:59.591660 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 245/491] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:43:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:43:59.592423 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da01a70-2b8b-4ed6-af87-a757ba4e0565 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:59.609937 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e29d924-5baa-44fe-b01b-228453b5dcd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:59.611381 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29d924-5baa-44fe-b01b-228453b5dcd1 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:59.612210 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29d924-5baa-44fe-b01b-228453b5dcd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:59.612667 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29d924-5baa-44fe-b01b-228453b5dcd1 None admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:59.613917 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e29d924-5baa-44fe-b01b-228453b5dcd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:59.614767 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e29d924-5baa-44fe-b01b-228453b5dcd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:59.616425 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e29d924-5baa-44fe-b01b-228453b5dcd1 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:59.634317 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 247/492] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:43:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:59.635690 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376607ff-6e52-44dd-a8b7-0e00cce88aec None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:59.635977 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376607ff-6e52-44dd-a8b7-0e00cce88aec None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:59.636253 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376607ff-6e52-44dd-a8b7-0e00cce88aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:59.652489 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-376607ff-6e52-44dd-a8b7-0e00cce88aec None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:43:59.653661 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-376607ff-6e52-44dd-a8b7-0e00cce88aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:43:59.683914 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:59.705320 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da01a70-2b8b-4ed6-af87-a757ba4e0565 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:59.735498 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 248/493] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:43:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:43:59.736322 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376607ff-6e52-44dd-a8b7-0e00cce88aec None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:59.746343 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3094a89d-e6e5-49fa-9d1f-538c4d742abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:59.747532 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094a89d-e6e5-49fa-9d1f-538c4d742abc None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:59.747913 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094a89d-e6e5-49fa-9d1f-538c4d742abc None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:59.748194 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094a89d-e6e5-49fa-9d1f-538c4d742abc None admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:59.749000 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3094a89d-e6e5-49fa-9d1f-538c4d742abc None admin] RBAC: Adding query filter params (name=service) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:43:59.749423 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3094a89d-e6e5-49fa-9d1f-538c4d742abc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:59.750852 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3094a89d-e6e5-49fa-9d1f-538c4d742abc None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:59.759599 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 246/494] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Jun 23 01:43:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:43:59.764434 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3094a89d-e6e5-49fa-9d1f-538c4d742abc None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:59.789191 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:43:59.841649 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376607ff-6e52-44dd-a8b7-0e00cce88aec None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:59.876009 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3094a89d-e6e5-49fa-9d1f-538c4d742abc None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:43:59.877774 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9ca824-d147-4eb0-9ec1-5812e9a88f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:59.878589 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ca824-d147-4eb0-9ec1-5812e9a88f6f None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:59.878909 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ca824-d147-4eb0-9ec1-5812e9a88f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:59.879187 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ca824-d147-4eb0-9ec1-5812e9a88f6f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:59.892219 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9ca824-d147-4eb0-9ec1-5812e9a88f6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:43:59.900029 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-1d9ca824-d147-4eb0-9ec1-5812e9a88f6f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 01:43:59.900029 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:59.900029 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:59.900029 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:59.900029 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:59.900029 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:59.900029 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:59.900029 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:59.900029 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:59.900029 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:59.900029 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:59.900029 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:59.900029 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:43:59.900029 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:43:59.900029 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:43:59.900029 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:43:59.900926 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:43:59.900926 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:43:59.900926 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:43:59.900926 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:43:59.900926 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:59.900926 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:59.900926 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:59.900926 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:59.900926 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:59.900926 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:59.900926 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:59.900926 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:59.900926 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:59.900926 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:59.900926 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:59.900926 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:59.901713 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:59.901713 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:59.901713 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:59.901713 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:59.901713 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:43:59.901713 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:43:59.901713 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:43:59.901713 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:43:59.901713 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:43:59.901713 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:43:59.901713 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 01:43:59.901713 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:43:59.901713 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 249/495] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jun 23 01:43:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:43:59.902348 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9ca824-d147-4eb0-9ec1-5812e9a88f6f None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:59.918240 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b11755-4a50-4de4-b141-2ff79a440b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:43:59.919571 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b11755-4a50-4de4-b141-2ff79a440b81 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:43:59.919927 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b11755-4a50-4de4-b141-2ff79a440b81 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:43:59.920201 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b11755-4a50-4de4-b141-2ff79a440b81 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:43:59.928456 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b11755-4a50-4de4-b141-2ff79a440b81 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:43:59.929917 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b11755-4a50-4de4-b141-2ff79a440b81 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:43:59.945348 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-e9b11755-4a50-4de4-b141-2ff79a440b81 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 01:43:59.945348 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:43:59.945348 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:43:59.945348 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:43:59.945348 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:43:59.945348 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:43:59.945348 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:43:59.945348 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:43:59.945348 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:43:59.945348 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:43:59.945348 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:43:59.945348 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:43:59.945348 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 01:43:59.945348 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 01:43:59.945348 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 01:43:59.945348 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 01:43:59.946261 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:43:59.946261 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:43:59.946261 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 01:43:59.946261 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:43:59.946261 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 01:43:59.946261 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:43:59.946261 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:43:59.946261 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:43:59.946261 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:43:59.946261 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:43:59.946261 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:43:59.946261 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:43:59.946261 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:43:59.946261 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:43:59.946261 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:43:59.946261 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:43:59.946261 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:43:59.946261 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:43:59.947137 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:43:59.947137 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:43:59.947137 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 01:43:59.947137 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 01:43:59.947137 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 01:43:59.947137 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 01:43:59.947137 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 01:43:59.947137 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 01:43:59.947137 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 01:43:59.947137 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:43:59.951007 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 247/496] 10.222.0.26 () {56 vars in 1093 bytes} [Thu Jun 23 01:43:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:43:59.951819 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b11755-4a50-4de4-b141-2ff79a440b81 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:43:59.955410 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:00.013478 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9ca824-d147-4eb0-9ec1-5812e9a88f6f None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:00.024080 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:00.048419 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfbf5e7-b18b-4b41-b90e-459532d8c57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:00.049245 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfbf5e7-b18b-4b41-b90e-459532d8c57c None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:00.049509 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfbf5e7-b18b-4b41-b90e-459532d8c57c None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:00.049785 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfbf5e7-b18b-4b41-b90e-459532d8c57c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:00.050637 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b11755-4a50-4de4-b141-2ff79a440b81 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:00.060161 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfbf5e7-b18b-4b41-b90e-459532d8c57c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:44:00.067852 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-dcfbf5e7-b18b-4b41-b90e-459532d8c57c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 01:44:00.067852 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:00.067852 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:00.067852 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:00.067852 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:00.067852 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:00.067852 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:00.067852 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:00.067852 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:00.067852 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:00.067852 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:00.067852 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:00.067852 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:44:00.067852 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:44:00.067852 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:44:00.067852 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:44:00.068779 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:44:00.068779 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:44:00.068779 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:44:00.068779 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:44:00.068779 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:00.068779 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:00.068779 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:00.068779 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:00.068779 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:00.068779 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:00.068779 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:00.068779 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:00.068779 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:00.068779 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:00.068779 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:00.068779 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:00.069557 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:00.069557 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:00.069557 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:00.069557 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:00.069557 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:44:00.069557 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:44:00.069557 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:44:00.069557 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:44:00.069557 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:44:00.069557 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:44:00.069557 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 01:44:00.069557 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:00.072553 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 250/497] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jun 23 01:43:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:00.072553 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfbf5e7-b18b-4b41-b90e-459532d8c57c None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:00.084393 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29542de4-53b0-45f8-8c4a-b481fd47c648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:00.085188 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29542de4-53b0-45f8-8c4a-b481fd47c648 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:00.085474 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29542de4-53b0-45f8-8c4a-b481fd47c648 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:00.085749 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29542de4-53b0-45f8-8c4a-b481fd47c648 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:00.096343 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29542de4-53b0-45f8-8c4a-b481fd47c648 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:00.097876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29542de4-53b0-45f8-8c4a-b481fd47c648 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:00.114757 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-29542de4-53b0-45f8-8c4a-b481fd47c648 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:00.114757 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:00.114757 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:00.114757 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:00.114757 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:00.114757 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:00.114757 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:00.114757 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:00.114757 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:00.114757 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:00.114757 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:00.114757 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:00.114757 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:00.114757 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:00.114757 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:00.115856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:00.115856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:00.115856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:00.115856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:00.115856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:00.115856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:00.115856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:00.115856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:00.115856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:00.115856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:00.115856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:00.115856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:00.115856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:00.115856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:00.115856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:00.115856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:00.115856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:00.117371 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:00.117371 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:00.117371 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:00.117371 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:00.117371 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 248/498] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:43:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:00.117371 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29542de4-53b0-45f8-8c4a-b481fd47c648 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:00.174768 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:00.180575 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfbf5e7-b18b-4b41-b90e-459532d8c57c None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:00.214186 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faed8654-349c-47ac-8be0-231fb7d63079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:00.214804 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed8654-349c-47ac-8be0-231fb7d63079 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:00.215087 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed8654-349c-47ac-8be0-231fb7d63079 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:00.215394 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed8654-349c-47ac-8be0-231fb7d63079 None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:00.216150 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faed8654-349c-47ac-8be0-231fb7d63079 None admin] RBAC: Adding query filter params (name=service) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:00.216444 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faed8654-349c-47ac-8be0-231fb7d63079 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:00.217825 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faed8654-349c-47ac-8be0-231fb7d63079 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:00.219062 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29542de4-53b0-45f8-8c4a-b481fd47c648 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:00.225893 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 251/499] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Jun 23 01:44:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:00.227128 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8360422-eaeb-4de4-9d38-b5b68472ff2c None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:00.227439 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8360422-eaeb-4de4-9d38-b5b68472ff2c None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:00.227718 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8360422-eaeb-4de4-9d38-b5b68472ff2c None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:00.231326 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 252/500] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:00.231326 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8360422-eaeb-4de4-9d38-b5b68472ff2c None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:00.255639 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5d3de5-2c03-48d3-a49b-52c3243ee93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:00.256302 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d3de5-2c03-48d3-a49b-52c3243ee93a None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:00.256603 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d3de5-2c03-48d3-a49b-52c3243ee93a None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:00.256878 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d3de5-2c03-48d3-a49b-52c3243ee93a None admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:00.257715 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5d3de5-2c03-48d3-a49b-52c3243ee93a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:00.258025 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5d3de5-2c03-48d3-a49b-52c3243ee93a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:00.259502 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5d3de5-2c03-48d3-a49b-52c3243ee93a None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:00.264110 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:00.303412 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 249/501] 10.222.0.26 () {56 vars in 1133 bytes} [Thu Jun 23 01:44:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:00.303412 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5d3de5-2c03-48d3-a49b-52c3243ee93a None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:00.342941 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8360422-eaeb-4de4-9d38-b5b68472ff2c None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:00.376559 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1a253b-38a2-493f-8746-ab1f7c8e7269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:00.377217 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a253b-38a2-493f-8746-ab1f7c8e7269 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:00.377494 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a253b-38a2-493f-8746-ab1f7c8e7269 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:00.377766 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a253b-38a2-493f-8746-ab1f7c8e7269 None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:00.378510 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1a253b-38a2-493f-8746-ab1f7c8e7269 None admin] RBAC: Adding query filter params (name=service) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:00.378819 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1a253b-38a2-493f-8746-ab1f7c8e7269 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:00.380224 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1a253b-38a2-493f-8746-ab1f7c8e7269 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:00.388053 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 253/502] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Jun 23 01:44:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:00.392429 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1a253b-38a2-493f-8746-ab1f7c8e7269 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:00.402021 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5d3de5-2c03-48d3-a49b-52c3243ee93a None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:00.417206 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:00.438075 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c27874-d19b-4747-a9fc-415273477fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:00.438734 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c27874-d19b-4747-a9fc-415273477fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:00.439019 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c27874-d19b-4747-a9fc-415273477fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:00.439336 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c27874-d19b-4747-a9fc-415273477fe3 None admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:00.440081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c27874-d19b-4747-a9fc-415273477fe3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:00.440390 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c27874-d19b-4747-a9fc-415273477fe3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:00.441794 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c27874-d19b-4747-a9fc-415273477fe3 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:00.452901 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 250/503] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:00.454193 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8a145-0ddf-4b54-a7e0-d9b7f226ba6e None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:00.454490 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8a145-0ddf-4b54-a7e0-d9b7f226ba6e None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:00.454771 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8a145-0ddf-4b54-a7e0-d9b7f226ba6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:00.470698 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-cdf8a145-0ddf-4b54-a7e0-d9b7f226ba6e None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:00.471872 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-cdf8a145-0ddf-4b54-a7e0-d9b7f226ba6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:00.486345 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:00.493855 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1a253b-38a2-493f-8746-ab1f7c8e7269 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:00.527165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e34148-6849-4527-a1c3-db90d6bcb32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:00.527986 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e34148-6849-4527-a1c3-db90d6bcb32f None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:00.528258 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e34148-6849-4527-a1c3-db90d6bcb32f None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:00.528525 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e34148-6849-4527-a1c3-db90d6bcb32f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:00.537412 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 251/504] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:44:00.538217 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf8a145-0ddf-4b54-a7e0-d9b7f226ba6e None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:00.539243 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e34148-6849-4527-a1c3-db90d6bcb32f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:00.540759 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e34148-6849-4527-a1c3-db90d6bcb32f None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:00.558591 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-67e34148-6849-4527-a1c3-db90d6bcb32f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:00.558591 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:00.558591 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:00.558591 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:00.558591 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:00.558591 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:00.558591 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:00.558591 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:00.558591 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:00.558591 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:00.558591 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:00.558591 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:00.558591 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:00.558591 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:00.558591 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:00.559613 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:00.559613 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:00.559613 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:00.559613 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:00.559613 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:00.559613 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:00.559613 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:00.559613 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:00.559613 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:00.559613 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:00.559613 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:00.559613 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:00.559613 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:00.559613 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:00.559613 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:00.559613 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:00.559613 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:00.560561 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:00.560561 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:00.560561 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:00.560561 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:00.560561 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 254/505] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:00.560860 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e34148-6849-4527-a1c3-db90d6bcb32f None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:00.649180 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf8a145-0ddf-4b54-a7e0-d9b7f226ba6e None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:00.651514 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:00.660477 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e34148-6849-4527-a1c3-db90d6bcb32f None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:00.684943 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a04527b-b678-4337-9ccd-146d2fbb5c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:00.685585 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a04527b-b678-4337-9ccd-146d2fbb5c9b None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:00.685868 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a04527b-b678-4337-9ccd-146d2fbb5c9b None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:00.686143 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a04527b-b678-4337-9ccd-146d2fbb5c9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:00.696304 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d77190b-c930-4dd5-88c4-b72efc63a239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:00.697101 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d77190b-c930-4dd5-88c4-b72efc63a239 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:00.697374 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d77190b-c930-4dd5-88c4-b72efc63a239 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:00.697643 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d77190b-c930-4dd5-88c4-b72efc63a239 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:00.708303 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d77190b-c930-4dd5-88c4-b72efc63a239 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:00.709777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d77190b-c930-4dd5-88c4-b72efc63a239 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:00.716484 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:00.727904 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-3d77190b-c930-4dd5-88c4-b72efc63a239 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:00.727904 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:00.727904 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:00.727904 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:00.727904 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:00.727904 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:00.727904 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:00.727904 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:00.727904 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:00.727904 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:00.727904 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:00.727904 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:00.727904 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:00.727904 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:00.727904 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:00.728866 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:00.728866 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:00.728866 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:00.728866 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:00.728866 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:00.728866 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:00.728866 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:00.728866 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:00.728866 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:00.728866 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:00.728866 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:00.728866 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:00.728866 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:00.728866 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:00.728866 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:00.728866 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:00.728866 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:00.729879 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:00.729879 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:00.729879 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:00.729879 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:00.729879 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 255/506] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:00.729879 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d77190b-c930-4dd5-88c4-b72efc63a239 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:00.758052 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a04527b-b678-4337-9ccd-146d2fbb5c9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:00.759586 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a04527b-b678-4337-9ccd-146d2fbb5c9b None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:00.829289 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d77190b-c930-4dd5-88c4-b72efc63a239 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:00.857138 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 252/507] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:44:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 01:44:00.858033 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a04527b-b678-4337-9ccd-146d2fbb5c9b None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:00.861249 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:00.863548 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee737e58-fe3d-4d75-b702-8d6deed55411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:00.864397 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee737e58-fe3d-4d75-b702-8d6deed55411 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:00.864677 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee737e58-fe3d-4d75-b702-8d6deed55411 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:00.864950 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee737e58-fe3d-4d75-b702-8d6deed55411 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:00.872678 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee737e58-fe3d-4d75-b702-8d6deed55411 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:00.874124 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee737e58-fe3d-4d75-b702-8d6deed55411 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:00.889324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-ee737e58-fe3d-4d75-b702-8d6deed55411 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 23 01:44:00.889324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:00.889324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:00.889324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:00.889324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:00.889324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:00.889324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:00.889324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:00.889324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:00.889324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:00.889324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:00.889324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:00.889324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 01:44:00.889324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 01:44:00.889324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 01:44:00.889324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 01:44:00.890187 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:00.890187 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:00.890187 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 01:44:00.890187 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:00.890187 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 01:44:00.890187 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:00.890187 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:00.890187 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:00.890187 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:00.890187 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:00.890187 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:00.890187 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:00.890187 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:00.890187 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:00.890187 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:00.890187 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:00.890187 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:00.890187 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:00.891039 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:00.891039 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:00.891039 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 01:44:00.891039 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 01:44:00.891039 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 01:44:00.891039 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 01:44:00.891039 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 01:44:00.891039 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 01:44:00.891039 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 23 01:44:00.891039 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:00.891039 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 256/508] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:44:00 2022] GET /identity/v3/users/glance => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:00.891614 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee737e58-fe3d-4d75-b702-8d6deed55411 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:00.948858 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:00.958835 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a04527b-b678-4337-9ccd-146d2fbb5c9b None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:00.989264 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee737e58-fe3d-4d75-b702-8d6deed55411 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:00.991420 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7de1204-39ff-4bd6-8094-89d9778103f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:00.992228 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de1204-39ff-4bd6-8094-89d9778103f1 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:00.992505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de1204-39ff-4bd6-8094-89d9778103f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:00.992748 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de1204-39ff-4bd6-8094-89d9778103f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:01.002594 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7de1204-39ff-4bd6-8094-89d9778103f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:01.004177 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7de1204-39ff-4bd6-8094-89d9778103f1 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:01.021596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-a7de1204-39ff-4bd6-8094-89d9778103f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:01.021596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:01.021596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:01.021596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:01.021596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:01.021596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:01.021596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:01.021596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:01.021596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:01.021596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:01.021596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:01.021596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:01.021596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:01.021596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:01.021596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:01.022442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:01.022442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:01.022442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:01.022442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:01.022442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:01.022442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:01.022442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:01.022442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:01.022442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:01.022442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:01.022442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:01.022442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:01.022442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:01.022442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:01.022442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:01.022442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:01.022442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:01.023369 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:01.023369 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:01.023369 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:01.023369 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:01.023369 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 253/509] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:01.023622 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7de1204-39ff-4bd6-8094-89d9778103f1 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:01.028178 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730eee3a-471c-403d-b95d-28ea079214d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:01.028867 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730eee3a-471c-403d-b95d-28ea079214d1 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:01.029152 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730eee3a-471c-403d-b95d-28ea079214d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:01.029427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730eee3a-471c-403d-b95d-28ea079214d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:01.030156 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730eee3a-471c-403d-b95d-28ea079214d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:01.030461 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730eee3a-471c-403d-b95d-28ea079214d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:01.031879 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730eee3a-471c-403d-b95d-28ea079214d1 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:01.044243 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 257/510] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:01.045041 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730eee3a-471c-403d-b95d-28ea079214d1 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:01.080461 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:01.124902 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7de1204-39ff-4bd6-8094-89d9778103f1 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:01.145791 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730eee3a-471c-403d-b95d-28ea079214d1 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:01.159057 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7464e65-f1e4-4142-a36a-bd793adbae34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:01.159746 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7464e65-f1e4-4142-a36a-bd793adbae34 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:01.160041 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7464e65-f1e4-4142-a36a-bd793adbae34 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:01.160325 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7464e65-f1e4-4142-a36a-bd793adbae34 None admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:01.161054 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7464e65-f1e4-4142-a36a-bd793adbae34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:01.161357 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7464e65-f1e4-4142-a36a-bd793adbae34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:01.162592 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:01.162739 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7464e65-f1e4-4142-a36a-bd793adbae34 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:01.178934 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 254/511] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:01.178934 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7464e65-f1e4-4142-a36a-bd793adbae34 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:01.184952 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465f4fb5-62ae-41c4-9eb4-379fe7ee5980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:01.185715 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465f4fb5-62ae-41c4-9eb4-379fe7ee5980 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:01.185995 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465f4fb5-62ae-41c4-9eb4-379fe7ee5980 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:01.186269 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465f4fb5-62ae-41c4-9eb4-379fe7ee5980 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:01.196956 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465f4fb5-62ae-41c4-9eb4-379fe7ee5980 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:01.198432 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465f4fb5-62ae-41c4-9eb4-379fe7ee5980 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:01.215895 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-465f4fb5-62ae-41c4-9eb4-379fe7ee5980 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:01.215895 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:01.215895 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:01.215895 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:01.215895 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:01.215895 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:01.215895 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:01.215895 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:01.215895 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:01.215895 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:01.215895 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:01.215895 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:01.215895 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:01.215895 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:01.215895 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:01.216742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:01.216742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:01.216742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:01.216742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:01.216742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:01.216742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:01.216742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:01.216742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:01.216742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:01.216742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:01.216742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:01.216742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:01.216742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:01.216742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:01.216742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:01.216742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:01.216742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:01.217571 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:01.217571 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:01.217571 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:01.217571 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:01.217571 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 258/512] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:01.217847 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465f4fb5-62ae-41c4-9eb4-379fe7ee5980 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:01.281284 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7464e65-f1e4-4142-a36a-bd793adbae34 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:01.297925 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:01.316507 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bce2bdc-a767-4501-af88-0483ce4bb565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:01.317520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce2bdc-a767-4501-af88-0483ce4bb565 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:01.317801 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce2bdc-a767-4501-af88-0483ce4bb565 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:01.318077 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce2bdc-a767-4501-af88-0483ce4bb565 None admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:01.318915 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bce2bdc-a767-4501-af88-0483ce4bb565 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:01.319244 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bce2bdc-a767-4501-af88-0483ce4bb565 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:01.320686 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bce2bdc-a767-4501-af88-0483ce4bb565 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:01.320840 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465f4fb5-62ae-41c4-9eb4-379fe7ee5980 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:01.337714 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 255/513] 10.222.0.26 () {56 vars in 1137 bytes} [Thu Jun 23 01:44:01 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:01.338509 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bce2bdc-a767-4501-af88-0483ce4bb565 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:01.354266 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed1909-3073-4297-9633-c0210370b194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:01.354941 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed1909-3073-4297-9633-c0210370b194 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:01.355235 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed1909-3073-4297-9633-c0210370b194 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:01.355528 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed1909-3073-4297-9633-c0210370b194 None admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:01.356263 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ed1909-3073-4297-9633-c0210370b194 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:01.356568 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ed1909-3073-4297-9633-c0210370b194 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:01.357954 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ed1909-3073-4297-9633-c0210370b194 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:01.369084 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 259/514] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:01.369699 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed1909-3073-4297-9633-c0210370b194 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:01.398222 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:01.435331 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bce2bdc-a767-4501-af88-0483ce4bb565 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:01.467956 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed1909-3073-4297-9633-c0210370b194 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:01.468462 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85fb74e-099a-4bd5-919c-3fe589beb485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:01.469235 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85fb74e-099a-4bd5-919c-3fe589beb485 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:01.469536 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85fb74e-099a-4bd5-919c-3fe589beb485 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:01.469802 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85fb74e-099a-4bd5-919c-3fe589beb485 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:01.476769 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85fb74e-099a-4bd5-919c-3fe589beb485 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:01.478271 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85fb74e-099a-4bd5-919c-3fe589beb485 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:01.492569 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-e85fb74e-099a-4bd5-919c-3fe589beb485 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 01:44:01.492569 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:01.492569 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:01.492569 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:01.492569 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:01.492569 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:01.492569 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:01.492569 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:01.492569 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:01.492569 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:01.492569 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:01.492569 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:01.492569 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 01:44:01.492569 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 01:44:01.492569 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 01:44:01.492569 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 01:44:01.493455 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:01.493455 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:01.493455 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 01:44:01.493455 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:01.493455 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 01:44:01.493455 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:01.493455 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:01.493455 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:01.493455 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:01.493455 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:01.493455 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:01.493455 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:01.493455 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:01.493455 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:01.493455 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:01.493455 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:01.493455 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:01.493455 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:01.494363 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:01.494363 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:01.494363 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 01:44:01.494363 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 01:44:01.494363 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 01:44:01.494363 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 01:44:01.494363 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 01:44:01.494363 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 01:44:01.494363 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 01:44:01.494363 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:01.495175 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 256/515] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:44:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:01.496172 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85fb74e-099a-4bd5-919c-3fe589beb485 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:01.500307 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfcd14e-338f-45d8-9ee2-27ce3fe6ec5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:01.501106 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfcd14e-338f-45d8-9ee2-27ce3fe6ec5c None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:01.501386 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfcd14e-338f-45d8-9ee2-27ce3fe6ec5c None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:01.501658 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfcd14e-338f-45d8-9ee2-27ce3fe6ec5c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:01.509317 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfcd14e-338f-45d8-9ee2-27ce3fe6ec5c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:01.510755 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfcd14e-338f-45d8-9ee2-27ce3fe6ec5c None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:01.524714 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-3dfcd14e-338f-45d8-9ee2-27ce3fe6ec5c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 01:44:01.524714 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:01.524714 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:01.524714 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:01.524714 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:01.524714 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:01.524714 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:01.524714 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:01.524714 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:01.524714 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:01.524714 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:01.524714 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:01.524714 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 01:44:01.524714 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 01:44:01.524714 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 01:44:01.524714 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 01:44:01.525786 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:01.525786 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:01.525786 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 01:44:01.525786 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:01.525786 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 01:44:01.525786 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:01.525786 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:01.525786 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:01.525786 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:01.525786 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:01.525786 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:01.525786 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:01.525786 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:01.525786 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:01.525786 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:01.525786 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:01.525786 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:01.525786 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:01.526867 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:01.526867 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:01.526867 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 01:44:01.526867 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 01:44:01.526867 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 01:44:01.526867 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 01:44:01.526867 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 01:44:01.526867 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 01:44:01.526867 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 01:44:01.526867 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:01.526867 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 260/516] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jun 23 01:44:01 2022] GET /identity/v3/users/neutron => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:01.526867 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfcd14e-338f-45d8-9ee2-27ce3fe6ec5c None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:01.532956 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:01.600865 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85fb74e-099a-4bd5-919c-3fe589beb485 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:01.615530 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:01.625725 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfcd14e-338f-45d8-9ee2-27ce3fe6ec5c None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:01.634123 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc822bc0-4081-4350-a7d2-1a5c74e5b736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:01.634768 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc822bc0-4081-4350-a7d2-1a5c74e5b736 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:01.635044 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc822bc0-4081-4350-a7d2-1a5c74e5b736 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:01.635342 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc822bc0-4081-4350-a7d2-1a5c74e5b736 None admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:01.636062 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc822bc0-4081-4350-a7d2-1a5c74e5b736 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:01.636368 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc822bc0-4081-4350-a7d2-1a5c74e5b736 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:01.637743 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc822bc0-4081-4350-a7d2-1a5c74e5b736 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:01.648944 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 257/517] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:01.649680 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc822bc0-4081-4350-a7d2-1a5c74e5b736 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:01.660311 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdea08df-cdd9-4b40-86c2-0fa844a654c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:01.660954 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea08df-cdd9-4b40-86c2-0fa844a654c7 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:01.661533 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea08df-cdd9-4b40-86c2-0fa844a654c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:01.661533 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea08df-cdd9-4b40-86c2-0fa844a654c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:01.727785 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdea08df-cdd9-4b40-86c2-0fa844a654c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:01.729239 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdea08df-cdd9-4b40-86c2-0fa844a654c7 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:01.741659 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:01.748905 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc822bc0-4081-4350-a7d2-1a5c74e5b736 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:01.782068 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a2062-6fb6-442f-b5ba-d4501eb11914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:01.782717 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a2062-6fb6-442f-b5ba-d4501eb11914 None admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:01.783056 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a2062-6fb6-442f-b5ba-d4501eb11914 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:01.783282 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a2062-6fb6-442f-b5ba-d4501eb11914 None admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:01.783980 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2a2062-6fb6-442f-b5ba-d4501eb11914 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:01.785461 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2a2062-6fb6-442f-b5ba-d4501eb11914 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:01.806634 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 258/518] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Jun 23 01:44:01 2022] POST /identity/v3/users => generated 247 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:44:01.807462 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a2062-6fb6-442f-b5ba-d4501eb11914 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:01.819434 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:01.834699 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 261/519] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:44:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 01:44:01.835996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdea08df-cdd9-4b40-86c2-0fa844a654c7 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:01.934026 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a2062-6fb6-442f-b5ba-d4501eb11914 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:01.962061 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdea08df-cdd9-4b40-86c2-0fa844a654c7 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:01.974196 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabc57f0-c48e-4e7e-af7e-143712fe272b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:01.974876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc57f0-c48e-4e7e-af7e-143712fe272b None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:01.975159 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc57f0-c48e-4e7e-af7e-143712fe272b None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:01.975499 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc57f0-c48e-4e7e-af7e-143712fe272b None admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:01.976352 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabc57f0-c48e-4e7e-af7e-143712fe272b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:01.977521 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabc57f0-c48e-4e7e-af7e-143712fe272b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:01.978104 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabc57f0-c48e-4e7e-af7e-143712fe272b None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:02.001064 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 259/520] 10.222.0.26 () {56 vars in 1137 bytes} [Thu Jun 23 01:44:01 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:02.001906 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabc57f0-c48e-4e7e-af7e-143712fe272b None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:02.008127 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021bb060-2726-459a-9c80-25be11a4f1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:02.008944 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021bb060-2726-459a-9c80-25be11a4f1ad None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:02.009325 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021bb060-2726-459a-9c80-25be11a4f1ad None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:02.009701 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021bb060-2726-459a-9c80-25be11a4f1ad None admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:02.010558 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021bb060-2726-459a-9c80-25be11a4f1ad None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:02.010867 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021bb060-2726-459a-9c80-25be11a4f1ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:02.012834 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021bb060-2726-459a-9c80-25be11a4f1ad None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:02.014354 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:02.053657 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 262/521] 10.222.0.26 () {56 vars in 1139 bytes} [Thu Jun 23 01:44:01 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:02.054415 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021bb060-2726-459a-9c80-25be11a4f1ad None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:02.107026 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:02.116361 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabc57f0-c48e-4e7e-af7e-143712fe272b None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:02.151163 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7907ae00-679d-405b-b957-bc6eba5e9dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:02.152075 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7907ae00-679d-405b-b957-bc6eba5e9dc4 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:02.152254 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7907ae00-679d-405b-b957-bc6eba5e9dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:02.152518 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7907ae00-679d-405b-b957-bc6eba5e9dc4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:02.159466 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7907ae00-679d-405b-b957-bc6eba5e9dc4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:02.160960 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7907ae00-679d-405b-b957-bc6eba5e9dc4 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:02.164875 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021bb060-2726-459a-9c80-25be11a4f1ad None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:02.175294 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-7907ae00-679d-405b-b957-bc6eba5e9dc4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:02.175294 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:02.175294 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:02.175294 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:02.175294 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:02.175294 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:02.175294 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:02.175294 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:02.175294 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:02.175294 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:02.175294 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:02.175294 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:02.175294 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 01:44:02.175294 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 01:44:02.175294 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 01:44:02.176264 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 01:44:02.176264 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:02.176264 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:02.176264 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:02.176264 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:02.176264 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:02.176264 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:02.176264 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:02.176264 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:02.176264 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:02.176264 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:02.176264 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:02.176264 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:02.176264 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:02.176264 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:02.176264 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:02.176264 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:02.177098 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 01:44:02.177098 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 01:44:02.177098 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 01:44:02.177098 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 01:44:02.177098 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 01:44:02.177098 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 01:44:02.177098 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:02.177098 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:02.177098 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 260/522] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:44:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:02.177593 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7907ae00-679d-405b-b957-bc6eba5e9dc4 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:02.205038 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8949c7-474c-4b5a-bb37-e2512e2a7704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:02.206307 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8949c7-474c-4b5a-bb37-e2512e2a7704 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:02.206781 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8949c7-474c-4b5a-bb37-e2512e2a7704 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:02.207336 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8949c7-474c-4b5a-bb37-e2512e2a7704 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:02.218594 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8949c7-474c-4b5a-bb37-e2512e2a7704 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:02.220467 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8949c7-474c-4b5a-bb37-e2512e2a7704 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:02.239610 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-7c8949c7-474c-4b5a-bb37-e2512e2a7704 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:02.239610 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:02.239610 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:02.239610 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:02.239610 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:02.239610 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:02.239610 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:02.239610 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:02.239610 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:02.239610 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:02.239610 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:02.239610 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:02.239610 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:02.239610 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:02.239610 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:02.240462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:02.240462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:02.240462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:02.240462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:02.240462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:02.240462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:02.240462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:02.240462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:02.240462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:02.240462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:02.240462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:02.240462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:02.240462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:02.240462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:02.240462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:02.240462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:02.240462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:02.241293 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:02.241293 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:02.241293 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:02.241293 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:02.243312 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 263/523] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:02.244231 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:02.245229 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8949c7-474c-4b5a-bb37-e2512e2a7704 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:02.289181 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7907ae00-679d-405b-b957-bc6eba5e9dc4 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:02.329904 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60130fc1-5c8a-4990-a018-390897a0777e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:02.330540 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60130fc1-5c8a-4990-a018-390897a0777e None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:02.330821 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60130fc1-5c8a-4990-a018-390897a0777e None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:02.331095 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60130fc1-5c8a-4990-a018-390897a0777e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:02.357178 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:02.361800 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8949c7-474c-4b5a-bb37-e2512e2a7704 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:02.400088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60130fc1-5c8a-4990-a018-390897a0777e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:02.401692 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60130fc1-5c8a-4990-a018-390897a0777e None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:02.402642 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa172aa2-8942-46fd-b887-b3e75876c6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:02.403328 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa172aa2-8942-46fd-b887-b3e75876c6a1 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:02.403760 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa172aa2-8942-46fd-b887-b3e75876c6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:02.404042 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa172aa2-8942-46fd-b887-b3e75876c6a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:02.470088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:02.485465 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa172aa2-8942-46fd-b887-b3e75876c6a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:02.488011 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa172aa2-8942-46fd-b887-b3e75876c6a1 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:02.507717 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 261/524] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:44:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 01:44:02.508572 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60130fc1-5c8a-4990-a018-390897a0777e None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:02.593519 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:02.598055 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 264/525] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:44:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 01:44:02.598884 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa172aa2-8942-46fd-b887-b3e75876c6a1 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:02.620329 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60130fc1-5c8a-4990-a018-390897a0777e None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:02.656412 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e4c5cf-c6bd-418d-bb9c-34132b1c1f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:02.657060 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e4c5cf-c6bd-418d-bb9c-34132b1c1f1e None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:02.657347 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e4c5cf-c6bd-418d-bb9c-34132b1c1f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:02.657627 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e4c5cf-c6bd-418d-bb9c-34132b1c1f1e None admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:02.658440 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e4c5cf-c6bd-418d-bb9c-34132b1c1f1e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:02.658741 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e4c5cf-c6bd-418d-bb9c-34132b1c1f1e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:02.660274 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e4c5cf-c6bd-418d-bb9c-34132b1c1f1e None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:02.678841 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 262/526] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jun 23 01:44:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:02.679832 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e4c5cf-c6bd-418d-bb9c-34132b1c1f1e None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:02.707860 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa172aa2-8942-46fd-b887-b3e75876c6a1 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:02.710070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:02.749586 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42870a7-6b3c-4859-9993-bf11960d9d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:02.750251 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42870a7-6b3c-4859-9993-bf11960d9d2e None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:02.750852 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42870a7-6b3c-4859-9993-bf11960d9d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:02.750852 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42870a7-6b3c-4859-9993-bf11960d9d2e None admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:02.751544 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42870a7-6b3c-4859-9993-bf11960d9d2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:02.751848 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42870a7-6b3c-4859-9993-bf11960d9d2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:02.753228 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42870a7-6b3c-4859-9993-bf11960d9d2e None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:02.764798 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 265/527] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:02.765607 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42870a7-6b3c-4859-9993-bf11960d9d2e None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:02.793240 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e4c5cf-c6bd-418d-bb9c-34132b1c1f1e None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:02.829111 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:02.831593 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bdeb56-654e-469e-951d-6b8ffea2536d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:02.832563 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bdeb56-654e-469e-951d-6b8ffea2536d None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:02.833260 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bdeb56-654e-469e-951d-6b8ffea2536d None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:02.833511 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bdeb56-654e-469e-951d-6b8ffea2536d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:02.843750 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bdeb56-654e-469e-951d-6b8ffea2536d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:02.845175 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bdeb56-654e-469e-951d-6b8ffea2536d None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:02.862596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-d3bdeb56-654e-469e-951d-6b8ffea2536d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:02.862596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:02.862596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:02.862596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:02.862596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:02.862596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:02.862596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:02.862596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:02.862596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:02.862596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:02.862596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:02.862596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:02.862596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:02.862596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:02.862596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:02.863577 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:02.863577 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:02.863577 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:02.863577 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:02.863577 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:02.863577 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:02.863577 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:02.863577 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:02.863577 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:02.863577 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:02.863577 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:02.863577 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:02.863577 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:02.863577 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:02.863577 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:02.863577 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:02.863577 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:02.864412 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:02.864412 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:02.864412 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:02.864412 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:02.868407 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 263/528] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:02.869014 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bdeb56-654e-469e-951d-6b8ffea2536d None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:02.875843 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42870a7-6b3c-4859-9993-bf11960d9d2e None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:02.909928 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00661e98-a535-4c74-9702-9a11bb54a7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:02.910752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00661e98-a535-4c74-9702-9a11bb54a7e1 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:02.911025 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00661e98-a535-4c74-9702-9a11bb54a7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:02.911326 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00661e98-a535-4c74-9702-9a11bb54a7e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:02.922698 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00661e98-a535-4c74-9702-9a11bb54a7e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:02.924221 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00661e98-a535-4c74-9702-9a11bb54a7e1 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:02.942996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-00661e98-a535-4c74-9702-9a11bb54a7e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:02.942996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:02.942996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:02.942996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:02.942996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:02.942996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:02.942996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:02.942996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:02.942996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:02.942996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:02.942996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:02.942996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:02.942996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:02.942996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:02.942996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:02.944254 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:02.944254 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:02.944254 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:02.944254 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:02.944254 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:02.944254 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:02.944254 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:02.944254 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:02.944254 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:02.944254 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:02.944254 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:02.944254 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:02.944254 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:02.944254 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:02.944254 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:02.944254 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:02.944254 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:02.945186 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:02.945186 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:02.945186 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:02.945186 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:02.945186 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 266/529] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:02.949388 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00661e98-a535-4c74-9702-9a11bb54a7e1 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:02.958841 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:02.980592 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bdeb56-654e-469e-951d-6b8ffea2536d None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:03.016948 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c0aa2-d91a-4329-a35e-f801b7da9608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:03.017731 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c0aa2-d91a-4329-a35e-f801b7da9608 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:03.018040 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c0aa2-d91a-4329-a35e-f801b7da9608 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:03.018383 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c0aa2-d91a-4329-a35e-f801b7da9608 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:03.019234 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23c0aa2-d91a-4329-a35e-f801b7da9608 None admin] RBAC: Adding query filter params (role.id=c20e054f00164967a4c16e7500c93361, scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c, user.id=ae4e83df33b64a3ebf28f54e2d94fe63) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:03.019575 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23c0aa2-d91a-4329-a35e-f801b7da9608 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:03.021033 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23c0aa2-d91a-4329-a35e-f801b7da9608 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:03.027175 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 264/530] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:44:02 2022] GET /identity/v3/role_assignments?user.id=ae4e83df33b64a3ebf28f54e2d94fe63&scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c&role.id=c20e054f00164967a4c16e7500c93361 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:03.027972 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c0aa2-d91a-4329-a35e-f801b7da9608 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:03.064341 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00661e98-a535-4c74-9702-9a11bb54a7e1 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:03.078639 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:03.103816 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d31f4a-09dd-43f8-ada3-0d47f7494a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:03.104712 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d31f4a-09dd-43f8-ada3-0d47f7494a76 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:03.104936 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d31f4a-09dd-43f8-ada3-0d47f7494a76 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:03.105212 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d31f4a-09dd-43f8-ada3-0d47f7494a76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:03.113030 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d31f4a-09dd-43f8-ada3-0d47f7494a76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:03.114557 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d31f4a-09dd-43f8-ada3-0d47f7494a76 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:03.133116 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-a6d31f4a-09dd-43f8-ada3-0d47f7494a76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:03.133116 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:03.133116 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:03.133116 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:03.133116 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:03.133116 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:03.133116 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:03.133116 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:03.133116 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:03.133116 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:03.133116 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:03.133116 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:03.133116 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 01:44:03.133116 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 01:44:03.133116 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 01:44:03.134040 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 01:44:03.134040 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:03.134040 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:03.134040 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:03.134040 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:03.134040 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:03.134040 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:03.134040 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:03.134040 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:03.134040 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:03.134040 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:03.134040 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:03.134040 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:03.134040 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:03.134040 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:03.134040 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:03.134040 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:03.134851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 01:44:03.134851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 01:44:03.134851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 01:44:03.134851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 01:44:03.134851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 01:44:03.134851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 01:44:03.134851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:03.134851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:03.134851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 267/531] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:44:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:03.135958 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d31f4a-09dd-43f8-ada3-0d47f7494a76 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:03.164057 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c0aa2-d91a-4329-a35e-f801b7da9608 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:03.208938 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23124639-f1f0-47d0-b1d0-7630609a8e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:03.210087 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23124639-f1f0-47d0-b1d0-7630609a8e36 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:03.210576 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23124639-f1f0-47d0-b1d0-7630609a8e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:03.211519 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23124639-f1f0-47d0-b1d0-7630609a8e36 None admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:03.212729 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23124639-f1f0-47d0-b1d0-7630609a8e36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:03.213242 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23124639-f1f0-47d0-b1d0-7630609a8e36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:03.215790 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23124639-f1f0-47d0-b1d0-7630609a8e36 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:03.236634 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 265/532] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:03.236634 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23124639-f1f0-47d0-b1d0-7630609a8e36 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:03.241674 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:03.250671 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d31f4a-09dd-43f8-ada3-0d47f7494a76 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:03.298134 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7dc27-3ab1-4dc7-880b-13ede9f9db4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:03.298847 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7dc27-3ab1-4dc7-880b-13ede9f9db4c None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:03.299546 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7dc27-3ab1-4dc7-880b-13ede9f9db4c None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:03.299831 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7dc27-3ab1-4dc7-880b-13ede9f9db4c None admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:03.300753 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f7dc27-3ab1-4dc7-880b-13ede9f9db4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:03.301175 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f7dc27-3ab1-4dc7-880b-13ede9f9db4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:03.302843 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f7dc27-3ab1-4dc7-880b-13ede9f9db4c None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:03.319284 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 268/533] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:03.320876 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7dc27-3ab1-4dc7-880b-13ede9f9db4c None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:03.374715 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:03.385616 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23124639-f1f0-47d0-b1d0-7630609a8e36 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:03.419999 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db246626-04ba-4a4d-8536-908e3b24c354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:03.421046 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db246626-04ba-4a4d-8536-908e3b24c354 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:03.421517 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db246626-04ba-4a4d-8536-908e3b24c354 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:03.421972 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db246626-04ba-4a4d-8536-908e3b24c354 None admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:03.422949 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db246626-04ba-4a4d-8536-908e3b24c354 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:03.423586 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db246626-04ba-4a4d-8536-908e3b24c354 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:03.425245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db246626-04ba-4a4d-8536-908e3b24c354 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:03.438121 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 266/534] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jun 23 01:44:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:03.439324 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db246626-04ba-4a4d-8536-908e3b24c354 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:03.451798 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7dc27-3ab1-4dc7-880b-13ede9f9db4c None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:03.489572 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade0352c-ded1-466b-b638-459db48dfce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:03.490497 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0352c-ded1-466b-b638-459db48dfce3 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:03.490784 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0352c-ded1-466b-b638-459db48dfce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:03.491059 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0352c-ded1-466b-b638-459db48dfce3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:03.500554 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade0352c-ded1-466b-b638-459db48dfce3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:03.502043 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade0352c-ded1-466b-b638-459db48dfce3 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:03.508343 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:03.517678 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-ade0352c-ded1-466b-b638-459db48dfce3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:03.517678 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:03.517678 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:03.517678 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:03.517678 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:03.517678 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:03.517678 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:03.517678 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:03.517678 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:03.517678 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:03.517678 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:03.517678 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:03.517678 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 01:44:03.517678 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 01:44:03.517678 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 01:44:03.518524 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 01:44:03.518524 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:03.518524 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:03.518524 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:03.518524 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:03.518524 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:03.518524 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:03.518524 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:03.518524 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:03.518524 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:03.518524 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:03.518524 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:03.518524 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:03.518524 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:03.518524 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:03.518524 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:03.518524 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:03.519870 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 01:44:03.519870 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 01:44:03.519870 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 01:44:03.519870 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 01:44:03.519870 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 01:44:03.519870 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 01:44:03.519870 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:03.519870 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:03.519870 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 269/535] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:44:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:03.520514 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade0352c-ded1-466b-b638-459db48dfce3 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:03.557708 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db246626-04ba-4a4d-8536-908e3b24c354 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:03.601720 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd06bc3-b0a2-4e4d-9f30-ed292dcd4a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:03.602825 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd06bc3-b0a2-4e4d-9f30-ed292dcd4a93 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:03.603112 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd06bc3-b0a2-4e4d-9f30-ed292dcd4a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:03.603491 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd06bc3-b0a2-4e4d-9f30-ed292dcd4a93 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:03.612255 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd06bc3-b0a2-4e4d-9f30-ed292dcd4a93 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:03.614260 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd06bc3-b0a2-4e4d-9f30-ed292dcd4a93 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:03.630310 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:03.634735 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-ddd06bc3-b0a2-4e4d-9f30-ed292dcd4a93 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:03.634735 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:03.634735 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:03.634735 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:03.634735 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:03.634735 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:03.634735 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:03.634735 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:03.634735 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:03.634735 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:03.634735 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:03.634735 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:03.634735 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 01:44:03.634735 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 01:44:03.634735 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 01:44:03.635838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 01:44:03.635838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:03.635838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:03.635838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:03.635838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:03.635838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:03.635838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:03.635838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:03.635838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:03.635838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:03.635838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:03.635838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:03.635838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:03.635838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:03.635838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:03.635838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:03.635838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:03.636695 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 01:44:03.636695 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 01:44:03.636695 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 01:44:03.636695 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 01:44:03.636695 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 01:44:03.636695 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 01:44:03.636695 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:03.636695 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:03.639279 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 267/536] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:44:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:03.640327 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd06bc3-b0a2-4e4d-9f30-ed292dcd4a93 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:03.647147 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade0352c-ded1-466b-b638-459db48dfce3 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:03.684207 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab06cf7e-2739-4429-8684-a43f7f30988b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:03.685226 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06cf7e-2739-4429-8684-a43f7f30988b None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:03.685226 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06cf7e-2739-4429-8684-a43f7f30988b None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:03.685425 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab06cf7e-2739-4429-8684-a43f7f30988b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:03.686249 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab06cf7e-2739-4429-8684-a43f7f30988b None admin] RBAC: Adding query filter params (role.id=c20e054f00164967a4c16e7500c93361, scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c, user.id=1c081b4cf34146ca8eb197b62a133433) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:03.686563 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab06cf7e-2739-4429-8684-a43f7f30988b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:03.688085 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab06cf7e-2739-4429-8684-a43f7f30988b None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:03.694436 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 270/537] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:44:03 2022] GET /identity/v3/role_assignments?user.id=1c081b4cf34146ca8eb197b62a133433&scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c&role.id=c20e054f00164967a4c16e7500c93361 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:03.695423 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab06cf7e-2739-4429-8684-a43f7f30988b None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:03.760243 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd06bc3-b0a2-4e4d-9f30-ed292dcd4a93 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:03.787040 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:03.802208 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a00d1-99e8-4045-abd9-4bda447e62a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:03.804346 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a00d1-99e8-4045-abd9-4bda447e62a8 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:03.804859 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a00d1-99e8-4045-abd9-4bda447e62a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:03.805645 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a00d1-99e8-4045-abd9-4bda447e62a8 None admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:03.806398 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab06cf7e-2739-4429-8684-a43f7f30988b None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:03.807359 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416a00d1-99e8-4045-abd9-4bda447e62a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:03.807916 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416a00d1-99e8-4045-abd9-4bda447e62a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:03.811102 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416a00d1-99e8-4045-abd9-4bda447e62a8 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:03.828204 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 268/538] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jun 23 01:44:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:03.840147 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a00d1-99e8-4045-abd9-4bda447e62a8 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:03.848656 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efca39b-7fea-4cdc-b0e0-9d667edacff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:03.849375 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efca39b-7fea-4cdc-b0e0-9d667edacff5 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:03.855344 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efca39b-7fea-4cdc-b0e0-9d667edacff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:03.855344 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efca39b-7fea-4cdc-b0e0-9d667edacff5 None admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:03.855344 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efca39b-7fea-4cdc-b0e0-9d667edacff5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:03.855344 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efca39b-7fea-4cdc-b0e0-9d667edacff5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:03.855344 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efca39b-7fea-4cdc-b0e0-9d667edacff5 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:03.873662 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 271/539] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jun 23 01:44:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:03.883918 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efca39b-7fea-4cdc-b0e0-9d667edacff5 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:03.899964 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:04.013621 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efca39b-7fea-4cdc-b0e0-9d667edacff5 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:04.026454 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a00d1-99e8-4045-abd9-4bda447e62a8 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:04.063745 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea0ed04-f642-4b1e-8028-38ff1329418c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:04.064445 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0ed04-f642-4b1e-8028-38ff1329418c None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:04.064729 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0ed04-f642-4b1e-8028-38ff1329418c None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:04.065010 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0ed04-f642-4b1e-8028-38ff1329418c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:04.065859 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea0ed04-f642-4b1e-8028-38ff1329418c None admin] RBAC: Adding query filter params (role.id=c20e054f00164967a4c16e7500c93361, scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c, user.id=948e4905de44473bb7a45e1b521cf095) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:04.066198 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea0ed04-f642-4b1e-8028-38ff1329418c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:04.066538 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a96d0d-2b1e-4526-b256-df30a9eddb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:04.067420 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a96d0d-2b1e-4526-b256-df30a9eddb7a None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:04.067704 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea0ed04-f642-4b1e-8028-38ff1329418c None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:04.068112 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a96d0d-2b1e-4526-b256-df30a9eddb7a None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:04.068589 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a96d0d-2b1e-4526-b256-df30a9eddb7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:04.069615 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a96d0d-2b1e-4526-b256-df30a9eddb7a None admin] RBAC: Adding query filter params (role.id=c20e054f00164967a4c16e7500c93361, scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c, user.id=dc5bb02f8eb64ac980e3218470b3c1b0) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:04.070116 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a96d0d-2b1e-4526-b256-df30a9eddb7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:04.071803 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a96d0d-2b1e-4526-b256-df30a9eddb7a None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:04.075359 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 272/540] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:44:03 2022] GET /identity/v3/role_assignments?user.id=948e4905de44473bb7a45e1b521cf095&scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c&role.id=c20e054f00164967a4c16e7500c93361 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:44:04.077025 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1322405d-e1e0-4077-ae6d-2cc0cb5c41dd None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:04.077580 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1322405d-e1e0-4077-ae6d-2cc0cb5c41dd None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:04.078045 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1322405d-e1e0-4077-ae6d-2cc0cb5c41dd None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:04.081218 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 273/541] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:04.094122 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3301eb-0b24-4588-b9f1-28e41a1dc5b9 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:04.094448 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3301eb-0b24-4588-b9f1-28e41a1dc5b9 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:04.094678 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3301eb-0b24-4588-b9f1-28e41a1dc5b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:04.097330 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 269/542] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:44:03 2022] GET /identity/v3/role_assignments?user.id=dc5bb02f8eb64ac980e3218470b3c1b0&scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c&role.id=c20e054f00164967a4c16e7500c93361 => generated 259 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:04.126435 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-7c3301eb-0b24-4588-b9f1-28e41a1dc5b9 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:04.128109 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-7c3301eb-0b24-4588-b9f1-28e41a1dc5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:04.241868 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 274/543] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:44:04.276240 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a96d0d-2b1e-4526-b256-df30a9eddb7a None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:04.336360 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:04.389030 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a96d0d-2b1e-4526-b256-df30a9eddb7a None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:04.436860 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a11b0c7-82c9-4548-87e6-e7d324bee89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:04.437668 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11b0c7-82c9-4548-87e6-e7d324bee89c None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:04.437957 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11b0c7-82c9-4548-87e6-e7d324bee89c None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:04.438238 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11b0c7-82c9-4548-87e6-e7d324bee89c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:04.453188 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a11b0c7-82c9-4548-87e6-e7d324bee89c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:44:04.461063 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-2a11b0c7-82c9-4548-87e6-e7d324bee89c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 01:44:04.461063 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:04.461063 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:04.461063 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:04.461063 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:04.461063 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:04.461063 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:04.461063 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:04.461063 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:04.461063 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:04.461063 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:04.461063 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:04.461063 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:44:04.461063 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:44:04.461063 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:44:04.461063 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:44:04.461915 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:44:04.461915 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:44:04.461915 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:44:04.461915 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:44:04.461915 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:04.461915 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:04.461915 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:04.461915 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:04.461915 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:04.461915 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:04.461915 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:04.461915 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:04.461915 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:04.461915 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:04.461915 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:04.461915 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:04.462677 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:04.462677 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:04.462677 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:04.462677 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:04.462677 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:44:04.462677 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:44:04.462677 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:44:04.462677 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:44:04.462677 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:44:04.462677 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:44:04.462677 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 01:44:04.462677 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:04.462677 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 270/544] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jun 23 01:44:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:04.469577 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3301eb-0b24-4588-b9f1-28e41a1dc5b9 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:04.481762 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:04.595533 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3301eb-0b24-4588-b9f1-28e41a1dc5b9 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:04.639997 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c9c05-c134-4798-ba8a-1d22f44c468f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:04.640676 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c9c05-c134-4798-ba8a-1d22f44c468f None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:04.640962 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c9c05-c134-4798-ba8a-1d22f44c468f None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:04.641241 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c9c05-c134-4798-ba8a-1d22f44c468f None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:04.642001 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9c9c05-c134-4798-ba8a-1d22f44c468f None admin] RBAC: Adding query filter params (name=service) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:04.642309 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9c9c05-c134-4798-ba8a-1d22f44c468f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:04.643740 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9c9c05-c134-4798-ba8a-1d22f44c468f None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:04.654307 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 275/545] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Jun 23 01:44:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:04.660089 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a11b0c7-82c9-4548-87e6-e7d324bee89c None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:04.782382 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a11b0c7-82c9-4548-87e6-e7d324bee89c None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:04.791794 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:04.817587 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6954b87-bc1b-4dd5-93bd-c4a6d6f2ce8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:04.818383 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6954b87-bc1b-4dd5-93bd-c4a6d6f2ce8d None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:04.818673 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6954b87-bc1b-4dd5-93bd-c4a6d6f2ce8d None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:04.818947 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6954b87-bc1b-4dd5-93bd-c4a6d6f2ce8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:04.829965 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6954b87-bc1b-4dd5-93bd-c4a6d6f2ce8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:04.831483 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6954b87-bc1b-4dd5-93bd-c4a6d6f2ce8d None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:04.849002 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-c6954b87-bc1b-4dd5-93bd-c4a6d6f2ce8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:04.849002 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:04.849002 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:04.849002 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:04.849002 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:04.849002 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:04.849002 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:04.849002 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:04.849002 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:04.849002 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:04.849002 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:04.849002 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:04.849002 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:04.849002 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:04.849002 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:04.849852 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:04.849852 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:04.849852 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:04.849852 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:04.849852 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:04.849852 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:04.849852 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:04.849852 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:04.849852 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:04.849852 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:04.849852 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:04.849852 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:04.849852 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:04.849852 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:04.849852 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:04.849852 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:04.849852 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:04.850661 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:04.850661 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:04.850661 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:04.850661 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:04.850661 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 271/546] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:04.855710 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c9c05-c134-4798-ba8a-1d22f44c468f None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:04.941473 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:04.984171 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c9c05-c134-4798-ba8a-1d22f44c468f None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:05.026924 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e0267c-8dcb-450d-9519-bca1b5ab35e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:05.027618 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0267c-8dcb-450d-9519-bca1b5ab35e6 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:05.027902 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0267c-8dcb-450d-9519-bca1b5ab35e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:05.028180 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0267c-8dcb-450d-9519-bca1b5ab35e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:05.028915 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e0267c-8dcb-450d-9519-bca1b5ab35e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:05.029226 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e0267c-8dcb-450d-9519-bca1b5ab35e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:05.030640 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e0267c-8dcb-450d-9519-bca1b5ab35e6 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:05.048862 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 276/547] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:44:05.054078 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6954b87-bc1b-4dd5-93bd-c4a6d6f2ce8d None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:05.208803 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6954b87-bc1b-4dd5-93bd-c4a6d6f2ce8d None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:05.257744 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c5e47-c3a6-43f8-aac6-b3c486c5031a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:05.258550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c5e47-c3a6-43f8-aac6-b3c486c5031a None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:05.258831 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c5e47-c3a6-43f8-aac6-b3c486c5031a None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:05.259106 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c5e47-c3a6-43f8-aac6-b3c486c5031a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:05.276302 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747c5e47-c3a6-43f8-aac6-b3c486c5031a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:05.277788 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747c5e47-c3a6-43f8-aac6-b3c486c5031a None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:05.299249 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-747c5e47-c3a6-43f8-aac6-b3c486c5031a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 01:44:05.299249 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:05.299249 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:05.299249 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:05.299249 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:05.299249 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:05.299249 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:05.299249 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:05.299249 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:05.299249 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:05.299249 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:05.299249 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:05.299249 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 01:44:05.299249 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 01:44:05.299249 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 01:44:05.299249 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 01:44:05.300225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:05.300225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:05.300225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 01:44:05.300225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:05.300225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 01:44:05.300225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:05.300225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:05.300225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:05.300225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:05.300225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:05.300225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:05.300225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:05.300225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:05.300225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:05.300225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:05.300225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:05.300225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:05.300225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:05.301642 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:05.301642 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:05.301642 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 01:44:05.301642 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 01:44:05.301642 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 01:44:05.301642 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 01:44:05.301642 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 01:44:05.301642 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 01:44:05.301642 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 01:44:05.301642 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:05.301642 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 272/548] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jun 23 01:44:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:05.303600 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e0267c-8dcb-450d-9519-bca1b5ab35e6 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:05.411615 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81225736-adc2-4f5e-8239-e8f8604968b0 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:05.411939 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81225736-adc2-4f5e-8239-e8f8604968b0 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:05.412169 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81225736-adc2-4f5e-8239-e8f8604968b0 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:05.413682 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 273/549] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 01:44:05.418440 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5d220-3ee6-42b0-893d-2f0eb14b070a None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:05.418959 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5d220-3ee6-42b0-893d-2f0eb14b070a None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:05.419459 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5d220-3ee6-42b0-893d-2f0eb14b070a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:05.435569 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e0267c-8dcb-450d-9519-bca1b5ab35e6 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:05.459122 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-bac5d220-3ee6-42b0-893d-2f0eb14b070a None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:05.460682 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-bac5d220-3ee6-42b0-893d-2f0eb14b070a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:05.467588 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:05.471820 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded4b643-5381-45ba-94a1-8c46c6e87c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:05.472584 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded4b643-5381-45ba-94a1-8c46c6e87c33 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:05.472902 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded4b643-5381-45ba-94a1-8c46c6e87c33 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:05.473208 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded4b643-5381-45ba-94a1-8c46c6e87c33 None admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:05.474117 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded4b643-5381-45ba-94a1-8c46c6e87c33 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:05.474450 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded4b643-5381-45ba-94a1-8c46c6e87c33 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:05.475966 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded4b643-5381-45ba-94a1-8c46c6e87c33 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:05.494379 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 277/550] 10.222.0.26 () {56 vars in 1135 bytes} [Thu Jun 23 01:44:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:05.499107 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:05.500472 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded4b643-5381-45ba-94a1-8c46c6e87c33 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:05.601603 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded4b643-5381-45ba-94a1-8c46c6e87c33 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:05.639407 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 274/551] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:44:05.647993 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b94500a-c778-44e2-8c8f-85e17469e09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:05.649062 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94500a-c778-44e2-8c8f-85e17469e09e None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:05.649552 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94500a-c778-44e2-8c8f-85e17469e09e None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:05.650045 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94500a-c778-44e2-8c8f-85e17469e09e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:05.661744 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac5d220-3ee6-42b0-893d-2f0eb14b070a None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:05.767260 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac5d220-3ee6-42b0-893d-2f0eb14b070a None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:05.803602 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:05.804605 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d444b65-e520-482f-be31-bdf6d31c6670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:05.805867 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d444b65-e520-482f-be31-bdf6d31c6670 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:05.806152 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d444b65-e520-482f-be31-bdf6d31c6670 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:05.806459 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d444b65-e520-482f-be31-bdf6d31c6670 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:05.818478 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d444b65-e520-482f-be31-bdf6d31c6670 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:44:05.826470 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b94500a-c778-44e2-8c8f-85e17469e09e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:05.826588 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-8d444b65-e520-482f-be31-bdf6d31c6670 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 01:44:05.826588 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:05.826588 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:05.826588 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:05.826588 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:05.826588 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:05.826588 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:05.826588 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:05.826588 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:05.826588 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:05.826588 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:05.826588 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:05.826588 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:44:05.826588 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:44:05.826588 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:44:05.826588 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:44:05.827473 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:44:05.827473 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:44:05.827473 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:44:05.827473 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:44:05.827473 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:05.827473 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:05.827473 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:05.827473 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:05.827473 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:05.827473 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:05.827473 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:05.827473 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:05.827473 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:05.827473 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:05.827473 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:05.827473 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:05.828375 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:05.828375 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:05.828375 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:05.828375 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:05.828375 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:44:05.828375 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:44:05.828375 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:44:05.828375 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:44:05.828375 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:44:05.828375 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:44:05.828375 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 01:44:05.828375 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:05.828375 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 275/552] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jun 23 01:44:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:44:05.831036 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b94500a-c778-44e2-8c8f-85e17469e09e None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:05.837190 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d444b65-e520-482f-be31-bdf6d31c6670 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:05.851599 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:05.953444 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d444b65-e520-482f-be31-bdf6d31c6670 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:05.989343 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53129f71-c5ff-4ca4-ae23-671464cafabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:05.990005 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53129f71-c5ff-4ca4-ae23-671464cafabf None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:05.990284 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53129f71-c5ff-4ca4-ae23-671464cafabf None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:05.990556 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53129f71-c5ff-4ca4-ae23-671464cafabf None admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:05.991337 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53129f71-c5ff-4ca4-ae23-671464cafabf None admin] RBAC: Adding query filter params (name=service) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:05.991640 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53129f71-c5ff-4ca4-ae23-671464cafabf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:05.993132 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53129f71-c5ff-4ca4-ae23-671464cafabf None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:06.002354 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 276/553] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Jun 23 01:44:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:06.006796 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 278/554] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:44:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 01:44:06.008393 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3bb18a-db75-461a-8c3f-f4bd5b5cb110 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:06.008682 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3bb18a-db75-461a-8c3f-f4bd5b5cb110 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:06.008959 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3bb18a-db75-461a-8c3f-f4bd5b5cb110 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:06.009160 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b94500a-c778-44e2-8c8f-85e17469e09e None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:06.010515 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 277/555] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:06.011728 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3bb18a-db75-461a-8c3f-f4bd5b5cb110 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:06.092326 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:06.104605 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:06.115283 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b94500a-c778-44e2-8c8f-85e17469e09e None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:06.127472 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3bb18a-db75-461a-8c3f-f4bd5b5cb110 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:06.151426 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00595158-541c-468a-acaf-ac38c2e10df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:06.152569 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00595158-541c-468a-acaf-ac38c2e10df6 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:06.153093 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00595158-541c-468a-acaf-ac38c2e10df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:06.153672 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00595158-541c-468a-acaf-ac38c2e10df6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:06.164378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac06204-ff82-46bf-aad6-a8a884849c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:06.165242 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac06204-ff82-46bf-aad6-a8a884849c42 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:06.165571 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac06204-ff82-46bf-aad6-a8a884849c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:06.165786 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac06204-ff82-46bf-aad6-a8a884849c42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:06.166936 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00595158-541c-468a-acaf-ac38c2e10df6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:06.168693 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00595158-541c-468a-acaf-ac38c2e10df6 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:06.177706 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac06204-ff82-46bf-aad6-a8a884849c42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:06.179383 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac06204-ff82-46bf-aad6-a8a884849c42 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:06.189259 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-00595158-541c-468a-acaf-ac38c2e10df6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:06.189259 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:06.189259 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:06.189259 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:06.189259 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:06.189259 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:06.189259 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:06.189259 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:06.189259 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:06.189259 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:06.189259 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:06.189259 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:06.189259 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:06.189259 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:06.189259 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:06.190135 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:06.190135 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:06.190135 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:06.190135 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:06.190135 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:06.190135 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:06.190135 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:06.190135 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:06.190135 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:06.190135 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:06.190135 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:06.190135 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:06.190135 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:06.190135 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:06.190135 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:06.190135 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:06.190135 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:06.190996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:06.190996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:06.190996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:06.190996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:06.193373 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 279/556] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:06.194798 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1f4bc6-6aa9-41c2-a097-60f5fc517a83 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:06.195310 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1f4bc6-6aa9-41c2-a097-60f5fc517a83 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:06.195847 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1f4bc6-6aa9-41c2-a097-60f5fc517a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:06.201821 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-9ac06204-ff82-46bf-aad6-a8a884849c42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:06.201821 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:06.201821 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:06.201821 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:06.201821 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:06.201821 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:06.201821 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:06.201821 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:06.201821 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:06.201821 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:06.201821 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:06.201821 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:06.201821 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:06.201821 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:06.201821 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:06.202651 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:06.202651 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:06.202651 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:06.202651 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:06.202651 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:06.202651 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:06.202651 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:06.202651 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:06.202651 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:06.202651 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:06.202651 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:06.202651 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:06.202651 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:06.202651 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:06.202651 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:06.202651 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:06.202651 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:06.204380 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:06.204380 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:06.204380 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:06.204380 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:06.209304 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 278/557] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:06.209801 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac06204-ff82-46bf-aad6-a8a884849c42 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:06.221731 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-8f1f4bc6-6aa9-41c2-a097-60f5fc517a83 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:06.222985 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-8f1f4bc6-6aa9-41c2-a097-60f5fc517a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:06.295047 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 280/558] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:44:06.295866 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1f4bc6-6aa9-41c2-a097-60f5fc517a83 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:06.346376 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac06204-ff82-46bf-aad6-a8a884849c42 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:06.376661 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:06.390235 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fda10bd-e3c6-46ab-8262-a2ba3188ca11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:06.390909 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda10bd-e3c6-46ab-8262-a2ba3188ca11 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:06.391200 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda10bd-e3c6-46ab-8262-a2ba3188ca11 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:06.391664 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda10bd-e3c6-46ab-8262-a2ba3188ca11 None admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:06.392546 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fda10bd-e3c6-46ab-8262-a2ba3188ca11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:06.392858 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fda10bd-e3c6-46ab-8262-a2ba3188ca11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:06.394458 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fda10bd-e3c6-46ab-8262-a2ba3188ca11 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:06.394915 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:06.413852 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 279/559] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:06.413852 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fda10bd-e3c6-46ab-8262-a2ba3188ca11 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:06.442759 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1f4bc6-6aa9-41c2-a097-60f5fc517a83 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:06.493522 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec306e44-1c2e-47cb-b4da-56774c585f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:06.494769 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec306e44-1c2e-47cb-b4da-56774c585f3d None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:06.495621 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec306e44-1c2e-47cb-b4da-56774c585f3d None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:06.495916 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec306e44-1c2e-47cb-b4da-56774c585f3d None admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:06.496678 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec306e44-1c2e-47cb-b4da-56774c585f3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:06.496988 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec306e44-1c2e-47cb-b4da-56774c585f3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:06.498429 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec306e44-1c2e-47cb-b4da-56774c585f3d None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:06.515997 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 281/560] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:06.515997 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec306e44-1c2e-47cb-b4da-56774c585f3d None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:06.548299 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fda10bd-e3c6-46ab-8262-a2ba3188ca11 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:06.584430 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fa4ad5-d896-47b5-b9a5-6206adc6efc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:06.585708 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa4ad5-d896-47b5-b9a5-6206adc6efc2 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:06.586180 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa4ad5-d896-47b5-b9a5-6206adc6efc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:06.586642 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa4ad5-d896-47b5-b9a5-6206adc6efc2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:06.597762 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fa4ad5-d896-47b5-b9a5-6206adc6efc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:44:06.606058 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-e9fa4ad5-d896-47b5-b9a5-6206adc6efc2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 01:44:06.606058 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:06.606058 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:06.606058 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:06.606058 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:06.606058 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:06.606058 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:06.606058 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:06.606058 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:06.606058 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:06.606058 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:06.606058 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:06.606058 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:44:06.606058 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:44:06.606058 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:44:06.606058 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:44:06.607028 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:44:06.607028 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:44:06.607028 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:44:06.607028 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:44:06.607028 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:06.607028 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:06.607028 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:06.607028 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:06.607028 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:06.607028 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:06.607028 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:06.607028 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:06.607028 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:06.607028 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:06.607028 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:06.607028 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:06.607881 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:06.607881 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:06.607881 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:06.607881 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:06.607881 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:44:06.607881 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:44:06.607881 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:44:06.607881 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:44:06.607881 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:44:06.607881 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:44:06.607881 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 01:44:06.607881 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:06.611349 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 280/561] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jun 23 01:44:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:06.612389 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fa4ad5-d896-47b5-b9a5-6206adc6efc2 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:06.620314 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec306e44-1c2e-47cb-b4da-56774c585f3d None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:06.652975 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:06.658454 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada4419-f621-45d2-9a1a-e3e1d8e92a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:06.659685 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada4419-f621-45d2-9a1a-e3e1d8e92a98 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:06.660422 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada4419-f621-45d2-9a1a-e3e1d8e92a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:06.660705 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada4419-f621-45d2-9a1a-e3e1d8e92a98 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:06.663911 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:06.669062 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ada4419-f621-45d2-9a1a-e3e1d8e92a98 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:06.670313 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ada4419-f621-45d2-9a1a-e3e1d8e92a98 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:06.684938 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-5ada4419-f621-45d2-9a1a-e3e1d8e92a98 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 01:44:06.684938 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:06.684938 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:06.684938 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:06.684938 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:06.684938 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:06.684938 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:06.684938 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:06.684938 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:06.684938 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:06.684938 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:06.684938 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:06.684938 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 01:44:06.684938 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 01:44:06.684938 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 01:44:06.684938 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 01:44:06.685952 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:06.685952 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:06.685952 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 01:44:06.685952 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:06.685952 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 01:44:06.685952 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:06.685952 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:06.685952 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:06.685952 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:06.685952 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:06.685952 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:06.685952 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:06.685952 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:06.685952 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:06.685952 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:06.685952 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:06.685952 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:06.685952 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:06.686844 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:06.686844 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:06.686844 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 01:44:06.686844 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 01:44:06.686844 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 01:44:06.686844 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 01:44:06.686844 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 01:44:06.686844 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 01:44:06.686844 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 01:44:06.686844 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:06.686844 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 282/562] 10.222.0.26 () {56 vars in 1093 bytes} [Thu Jun 23 01:44:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:06.688068 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd2c2d-10bd-4727-a21b-5c093918b648 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:06.688068 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd2c2d-10bd-4727-a21b-5c093918b648 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:06.688068 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd2c2d-10bd-4727-a21b-5c093918b648 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:06.689291 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 283/563] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:06.689873 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fd2c2d-10bd-4727-a21b-5c093918b648 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:06.729778 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fa4ad5-d896-47b5-b9a5-6206adc6efc2 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:06.764995 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc0bccd-2358-4f77-9526-cde26740c4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:06.765842 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0bccd-2358-4f77-9526-cde26740c4dc None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:06.766130 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0bccd-2358-4f77-9526-cde26740c4dc None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:06.766404 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0bccd-2358-4f77-9526-cde26740c4dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:06.773401 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc0bccd-2358-4f77-9526-cde26740c4dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:06.774886 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc0bccd-2358-4f77-9526-cde26740c4dc None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:06.789472 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-bcc0bccd-2358-4f77-9526-cde26740c4dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:06.789472 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:06.789472 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:06.789472 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:06.789472 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:06.789472 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:06.789472 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:06.789472 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:06.789472 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:06.789472 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:06.789472 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:06.789472 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:06.789472 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 01:44:06.789472 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 01:44:06.789472 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 01:44:06.790574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 01:44:06.790574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:06.790574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:06.790574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:06.790574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:06.790574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:06.790574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:06.790574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:06.790574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:06.790574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:06.790574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:06.790574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:06.790574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:06.790574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:06.790574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:06.790574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:06.790574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:06.791621 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 01:44:06.791621 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 01:44:06.791621 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 01:44:06.791621 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 01:44:06.791621 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 01:44:06.791621 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 01:44:06.791621 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:06.791621 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:06.791621 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 281/564] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:44:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:06.791621 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc0bccd-2358-4f77-9526-cde26740c4dc None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:06.798321 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fd2c2d-10bd-4727-a21b-5c093918b648 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:06.832891 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4ef0af-f21b-4d05-8129-bcd58c122896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:06.833553 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ef0af-f21b-4d05-8129-bcd58c122896 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:06.833830 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ef0af-f21b-4d05-8129-bcd58c122896 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:06.834103 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ef0af-f21b-4d05-8129-bcd58c122896 None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:06.834845 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4ef0af-f21b-4d05-8129-bcd58c122896 None admin] RBAC: Adding query filter params (name=service) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:06.835146 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4ef0af-f21b-4d05-8129-bcd58c122896 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:06.836552 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4ef0af-f21b-4d05-8129-bcd58c122896 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:06.844420 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 284/565] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Jun 23 01:44:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:06.845570 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64876a80-0a1f-41dd-abce-5a6522ce8f6e None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:06.845854 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64876a80-0a1f-41dd-abce-5a6522ce8f6e None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:06.846132 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64876a80-0a1f-41dd-abce-5a6522ce8f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:06.861882 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-64876a80-0a1f-41dd-abce-5a6522ce8f6e None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:06.863054 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-64876a80-0a1f-41dd-abce-5a6522ce8f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:06.891945 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc0bccd-2358-4f77-9526-cde26740c4dc None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:06.894492 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:06.912116 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:06.927077 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd5298-ad6a-4a20-853a-c774b9ffaeea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:06.927773 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd5298-ad6a-4a20-853a-c774b9ffaeea None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:06.928055 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd5298-ad6a-4a20-853a-c774b9ffaeea None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:06.928329 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd5298-ad6a-4a20-853a-c774b9ffaeea None admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:06.929173 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd5298-ad6a-4a20-853a-c774b9ffaeea None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:06.929278 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 285/566] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:44:06.929514 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd5298-ad6a-4a20-853a-c774b9ffaeea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:06.930815 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5fa252-d48b-4fd2-8d01-98debde10d7c None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:06.931103 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5fa252-d48b-4fd2-8d01-98debde10d7c None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:06.931260 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd5298-ad6a-4a20-853a-c774b9ffaeea None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:06.931363 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5fa252-d48b-4fd2-8d01-98debde10d7c None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:06.935261 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 286/567] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:06.935261 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5fa252-d48b-4fd2-8d01-98debde10d7c None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:06.951938 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 282/568] 10.222.0.26 () {56 vars in 1133 bytes} [Thu Jun 23 01:44:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:06.952759 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd5298-ad6a-4a20-853a-c774b9ffaeea None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:07.037344 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5fa252-d48b-4fd2-8d01-98debde10d7c None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:07.050828 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd5298-ad6a-4a20-853a-c774b9ffaeea None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:07.071462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc13fa7-0a55-4d26-a578-36c44688932e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:07.072123 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc13fa7-0a55-4d26-a578-36c44688932e None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:07.072405 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc13fa7-0a55-4d26-a578-36c44688932e None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:07.072680 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc13fa7-0a55-4d26-a578-36c44688932e None admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:07.073475 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc13fa7-0a55-4d26-a578-36c44688932e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:07.073784 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc13fa7-0a55-4d26-a578-36c44688932e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:07.075262 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc13fa7-0a55-4d26-a578-36c44688932e None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:07.083591 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59890c3a-7f2f-411f-88bf-1896657ec662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:07.084369 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59890c3a-7f2f-411f-88bf-1896657ec662 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:07.084646 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59890c3a-7f2f-411f-88bf-1896657ec662 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:07.084917 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59890c3a-7f2f-411f-88bf-1896657ec662 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:07.086958 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 287/569] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jun 23 01:44:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:07.088258 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda72c52-fbe2-4e2f-b30e-64b3327c7b4e None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:07.088619 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda72c52-fbe2-4e2f-b30e-64b3327c7b4e None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:07.088970 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda72c52-fbe2-4e2f-b30e-64b3327c7b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:07.095861 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59890c3a-7f2f-411f-88bf-1896657ec662 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:07.097352 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59890c3a-7f2f-411f-88bf-1896657ec662 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:07.104812 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-eda72c52-fbe2-4e2f-b30e-64b3327c7b4e None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:07.106014 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-eda72c52-fbe2-4e2f-b30e-64b3327c7b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:07.115116 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-59890c3a-7f2f-411f-88bf-1896657ec662 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:07.115116 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:07.115116 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:07.115116 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:07.115116 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:07.115116 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:07.115116 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:07.115116 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:07.115116 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:07.115116 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:07.115116 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:07.115116 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:07.115116 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:07.115116 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:07.115116 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:07.116177 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:07.116177 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:07.116177 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:07.116177 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:07.116177 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:07.116177 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:07.116177 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:07.116177 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:07.116177 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:07.116177 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:07.116177 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:07.116177 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:07.116177 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:07.116177 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:07.116177 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:07.116177 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:07.116177 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:07.117228 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:07.117228 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:07.117228 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:07.117228 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:07.119500 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 283/570] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:07.120372 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59890c3a-7f2f-411f-88bf-1896657ec662 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:07.135577 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:07.146127 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:07.173425 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 288/571] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:44:07.174253 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda72c52-fbe2-4e2f-b30e-64b3327c7b4e None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:07.223693 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59890c3a-7f2f-411f-88bf-1896657ec662 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:07.257146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7788d0-0cac-47b9-9bce-29bcbdd0a363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:07.257914 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7788d0-0cac-47b9-9bce-29bcbdd0a363 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:07.258227 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7788d0-0cac-47b9-9bce-29bcbdd0a363 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:07.258500 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7788d0-0cac-47b9-9bce-29bcbdd0a363 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:07.268895 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7788d0-0cac-47b9-9bce-29bcbdd0a363 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:44:07.272257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda72c52-fbe2-4e2f-b30e-64b3327c7b4e None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:07.276755 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-fe7788d0-0cac-47b9-9bce-29bcbdd0a363 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 01:44:07.276755 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:07.276755 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:07.276755 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:07.276755 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:07.276755 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:07.276755 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:07.276755 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:07.276755 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:07.276755 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:07.276755 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:07.276755 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:07.276755 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:44:07.276755 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:44:07.276755 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:44:07.276755 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:44:07.277823 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:44:07.277823 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:44:07.277823 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:44:07.277823 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:44:07.277823 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:07.277823 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:07.277823 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:07.277823 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:07.277823 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:07.277823 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:07.277823 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:07.277823 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:07.277823 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:07.277823 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:07.277823 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:07.277823 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:07.283767 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:07.283767 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:07.283767 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:07.283767 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:07.283767 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:44:07.283767 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:44:07.283767 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:44:07.283767 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:44:07.283767 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:44:07.283767 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:44:07.283767 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 01:44:07.283767 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:07.283767 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 284/572] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jun 23 01:44:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:07.283767 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7788d0-0cac-47b9-9bce-29bcbdd0a363 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:07.312412 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7391097b-c6f5-456a-988c-1d54d796de2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:07.313222 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7391097b-c6f5-456a-988c-1d54d796de2d None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:07.313500 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7391097b-c6f5-456a-988c-1d54d796de2d None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:07.313772 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7391097b-c6f5-456a-988c-1d54d796de2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:07.323979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7391097b-c6f5-456a-988c-1d54d796de2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:07.325436 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7391097b-c6f5-456a-988c-1d54d796de2d None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:07.344802 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-7391097b-c6f5-456a-988c-1d54d796de2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:07.344802 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:07.344802 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:07.344802 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:07.344802 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:07.344802 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:07.344802 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:07.344802 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:07.344802 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:07.344802 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:07.344802 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:07.344802 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:07.344802 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:07.344802 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:07.344802 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:07.345784 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:07.345784 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:07.345784 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:07.345784 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:07.345784 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:07.345784 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:07.345784 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:07.345784 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:07.345784 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:07.345784 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:07.345784 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:07.345784 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:07.345784 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:07.345784 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:07.345784 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:07.345784 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:07.345784 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:07.351742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:07.351742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:07.351742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:07.351742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:07.351742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 289/573] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:07.351742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7391097b-c6f5-456a-988c-1d54d796de2d None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:07.356646 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:07.375032 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:07.382399 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7788d0-0cac-47b9-9bce-29bcbdd0a363 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:07.415396 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a81e43-f18b-407c-9c7b-100d6ef77d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:07.416050 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a81e43-f18b-407c-9c7b-100d6ef77d8e None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:07.416333 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a81e43-f18b-407c-9c7b-100d6ef77d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:07.416605 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a81e43-f18b-407c-9c7b-100d6ef77d8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:07.417443 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a81e43-f18b-407c-9c7b-100d6ef77d8e None admin] RBAC: Adding query filter params (role.id=c20e054f00164967a4c16e7500c93361, scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c, user.id=28e59688485241c4b6d51f99bbb3d059) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:07.417753 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a81e43-f18b-407c-9c7b-100d6ef77d8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:07.419423 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a81e43-f18b-407c-9c7b-100d6ef77d8e None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:07.424885 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 285/574] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:44:07 2022] GET /identity/v3/role_assignments?user.id=28e59688485241c4b6d51f99bbb3d059&scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c&role.id=c20e054f00164967a4c16e7500c93361 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:07.425701 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a81e43-f18b-407c-9c7b-100d6ef77d8e None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:07.452168 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7391097b-c6f5-456a-988c-1d54d796de2d None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:07.491373 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b88f2c5-420f-433c-bef5-290328ab1906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:07.492217 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b88f2c5-420f-433c-bef5-290328ab1906 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:07.492595 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b88f2c5-420f-433c-bef5-290328ab1906 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:07.492972 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b88f2c5-420f-433c-bef5-290328ab1906 None admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:07.493819 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b88f2c5-420f-433c-bef5-290328ab1906 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:07.494205 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b88f2c5-420f-433c-bef5-290328ab1906 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:07.495786 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b88f2c5-420f-433c-bef5-290328ab1906 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:07.512660 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 290/575] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:07.512660 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b88f2c5-420f-433c-bef5-290328ab1906 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:07.542057 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a81e43-f18b-407c-9c7b-100d6ef77d8e None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:07.583675 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654df36b-0e57-43f6-bd3d-ea9710da3f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:07.584468 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654df36b-0e57-43f6-bd3d-ea9710da3f2b None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:07.584751 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654df36b-0e57-43f6-bd3d-ea9710da3f2b None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:07.585019 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654df36b-0e57-43f6-bd3d-ea9710da3f2b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:07.599631 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:07.604608 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654df36b-0e57-43f6-bd3d-ea9710da3f2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:44:07.605331 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:07.614854 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-654df36b-0e57-43f6-bd3d-ea9710da3f2b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 01:44:07.614854 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:07.614854 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:07.614854 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:07.614854 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:07.614854 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:07.614854 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:07.614854 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:07.614854 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:07.614854 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:07.614854 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:07.614854 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:07.614854 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:44:07.614854 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:44:07.614854 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:44:07.614854 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:44:07.615993 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:44:07.615993 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:44:07.615993 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:44:07.615993 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:44:07.615993 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:07.615993 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:07.615993 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:07.615993 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:07.615993 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:07.615993 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:07.615993 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:07.615993 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:07.615993 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:07.615993 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:07.615993 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:07.615993 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:07.617017 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:07.617017 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:07.617017 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:07.617017 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:07.617017 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:44:07.617017 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:44:07.617017 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:44:07.617017 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:44:07.617017 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:44:07.617017 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:44:07.617017 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 01:44:07.617017 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:07.617017 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 286/576] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jun 23 01:44:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:07.617847 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654df36b-0e57-43f6-bd3d-ea9710da3f2b None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:07.623966 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b88f2c5-420f-433c-bef5-290328ab1906 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:07.658468 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a1433a-3075-4c36-9b53-1b2509a5030b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:07.659171 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1433a-3075-4c36-9b53-1b2509a5030b None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:07.659549 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1433a-3075-4c36-9b53-1b2509a5030b None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:07.659939 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1433a-3075-4c36-9b53-1b2509a5030b None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:07.660787 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a1433a-3075-4c36-9b53-1b2509a5030b None admin] RBAC: Adding query filter params (name=service) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:07.661154 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a1433a-3075-4c36-9b53-1b2509a5030b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:07.662592 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a1433a-3075-4c36-9b53-1b2509a5030b None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:07.675500 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 291/577] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Jun 23 01:44:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:07.675500 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a1433a-3075-4c36-9b53-1b2509a5030b None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:07.739516 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654df36b-0e57-43f6-bd3d-ea9710da3f2b None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:07.775398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21108080-2f36-498d-9610-318dfc51cd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:07.776159 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21108080-2f36-498d-9610-318dfc51cd63 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:07.776496 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21108080-2f36-498d-9610-318dfc51cd63 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:07.776896 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21108080-2f36-498d-9610-318dfc51cd63 None admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:07.777709 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21108080-2f36-498d-9610-318dfc51cd63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:07.778073 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21108080-2f36-498d-9610-318dfc51cd63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:07.779583 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21108080-2f36-498d-9610-318dfc51cd63 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:07.779925 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a1433a-3075-4c36-9b53-1b2509a5030b None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:07.796973 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 287/578] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:07.797922 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21108080-2f36-498d-9610-318dfc51cd63 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:07.814990 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bc500c-fe3c-4182-9ea2-baa21cfecd1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:07.815904 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc500c-fe3c-4182-9ea2-baa21cfecd1e None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:07.816284 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc500c-fe3c-4182-9ea2-baa21cfecd1e None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:07.816690 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc500c-fe3c-4182-9ea2-baa21cfecd1e None admin] PATH_INFO: `/v3/users/glance` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:07.825763 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bc500c-fe3c-4182-9ea2-baa21cfecd1e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:07.827240 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bc500c-fe3c-4182-9ea2-baa21cfecd1e None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:07.841796 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-44bc500c-fe3c-4182-9ea2-baa21cfecd1e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 23 01:44:07.841796 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:07.841796 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:07.841796 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:07.841796 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:07.841796 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:07.841796 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:07.841796 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:07.841796 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:07.841796 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:07.841796 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:07.841796 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:07.841796 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 01:44:07.841796 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 01:44:07.841796 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 01:44:07.841796 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 01:44:07.842766 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:07.842766 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:07.842766 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 01:44:07.842766 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:07.842766 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 01:44:07.842766 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:07.842766 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:07.842766 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:07.842766 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:07.842766 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:07.842766 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:07.842766 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:07.842766 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:07.842766 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:07.842766 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:07.842766 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:07.842766 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:07.842766 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:07.844120 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:07.844120 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:07.844120 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 01:44:07.844120 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 01:44:07.844120 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 01:44:07.844120 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 01:44:07.844120 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 01:44:07.844120 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 01:44:07.844120 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 23 01:44:07.844120 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:07.844120 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 292/579] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:44:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:07.844120 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bc500c-fe3c-4182-9ea2-baa21cfecd1e None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:07.857549 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:07.863354 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:07.909904 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21108080-2f36-498d-9610-318dfc51cd63 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:07.947252 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408768a6-3386-40b2-88d3-9f3ce3fab5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:07.947936 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408768a6-3386-40b2-88d3-9f3ce3fab5e8 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:07.948217 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408768a6-3386-40b2-88d3-9f3ce3fab5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:07.948496 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408768a6-3386-40b2-88d3-9f3ce3fab5e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:07.949271 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408768a6-3386-40b2-88d3-9f3ce3fab5e8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:07.949583 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408768a6-3386-40b2-88d3-9f3ce3fab5e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:07.950969 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408768a6-3386-40b2-88d3-9f3ce3fab5e8 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:07.958416 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bc500c-fe3c-4182-9ea2-baa21cfecd1e None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:07.959252 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 288/580] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Jun 23 01:44:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:07.960053 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408768a6-3386-40b2-88d3-9f3ce3fab5e8 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:08.001537 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322fc509-875e-4277-a757-1aab1b0c7bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:08.002347 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322fc509-875e-4277-a757-1aab1b0c7bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:08.003034 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322fc509-875e-4277-a757-1aab1b0c7bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:08.003382 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322fc509-875e-4277-a757-1aab1b0c7bc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:08.016326 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322fc509-875e-4277-a757-1aab1b0c7bc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:08.017804 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322fc509-875e-4277-a757-1aab1b0c7bc2 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:08.036580 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-322fc509-875e-4277-a757-1aab1b0c7bc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:08.036580 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:08.036580 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:08.036580 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:08.036580 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:08.036580 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:08.036580 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:08.036580 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:08.036580 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:08.036580 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:08.036580 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:08.036580 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:08.036580 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:08.036580 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:08.036580 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:08.037958 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:08.037958 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:08.037958 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:08.037958 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:08.037958 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:08.037958 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:08.037958 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:08.037958 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:08.037958 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:08.037958 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:08.037958 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:08.037958 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:08.037958 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:08.037958 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:08.037958 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:08.037958 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:08.037958 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:08.038913 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:08.038913 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:08.038913 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:08.038913 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:08.038913 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 293/581] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:08.040073 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322fc509-875e-4277-a757-1aab1b0c7bc2 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:08.067624 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408768a6-3386-40b2-88d3-9f3ce3fab5e8 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:08.090076 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:08.102402 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:08.106632 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe81eb-1d93-48ee-a717-5923a8082860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:08.107467 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe81eb-1d93-48ee-a717-5923a8082860 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:08.107745 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe81eb-1d93-48ee-a717-5923a8082860 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:08.108019 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe81eb-1d93-48ee-a717-5923a8082860 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:08.114807 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fe81eb-1d93-48ee-a717-5923a8082860 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:08.116296 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fe81eb-1d93-48ee-a717-5923a8082860 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:08.132766 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-a6fe81eb-1d93-48ee-a717-5923a8082860 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:08.132766 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:08.132766 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:08.132766 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:08.132766 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:08.132766 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:08.132766 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:08.132766 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:08.132766 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:08.132766 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:08.132766 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:08.132766 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:08.132766 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 01:44:08.132766 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 01:44:08.132766 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 01:44:08.133792 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 01:44:08.133792 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:08.133792 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:08.133792 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:08.133792 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:08.133792 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:08.133792 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:08.133792 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:08.133792 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:08.133792 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:08.133792 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:08.133792 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:08.133792 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:08.133792 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:08.133792 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:08.133792 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:08.133792 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:08.134878 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 01:44:08.134878 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 01:44:08.134878 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 01:44:08.134878 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 01:44:08.134878 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 01:44:08.134878 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 01:44:08.134878 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:08.134878 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:08.136340 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 289/582] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:44:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:08.137150 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe81eb-1d93-48ee-a717-5923a8082860 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:08.148082 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322fc509-875e-4277-a757-1aab1b0c7bc2 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:08.188620 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2a2f4-8afe-4750-8486-76306e107b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:08.189421 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2a2f4-8afe-4750-8486-76306e107b84 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:08.189765 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2a2f4-8afe-4750-8486-76306e107b84 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:08.190213 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2a2f4-8afe-4750-8486-76306e107b84 None admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:08.191286 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b2a2f4-8afe-4750-8486-76306e107b84 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:08.191633 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b2a2f4-8afe-4750-8486-76306e107b84 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:08.193191 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b2a2f4-8afe-4750-8486-76306e107b84 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:08.216707 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 294/583] 10.222.0.26 () {56 vars in 1137 bytes} [Thu Jun 23 01:44:08 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:08.216707 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2a2f4-8afe-4750-8486-76306e107b84 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:08.242509 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe81eb-1d93-48ee-a717-5923a8082860 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:08.279194 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa893b18-04e2-4d67-8ae4-753d060a2dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:08.280421 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa893b18-04e2-4d67-8ae4-753d060a2dd5 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:08.280994 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa893b18-04e2-4d67-8ae4-753d060a2dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:08.281334 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa893b18-04e2-4d67-8ae4-753d060a2dd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:08.292381 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa893b18-04e2-4d67-8ae4-753d060a2dd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:08.294026 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa893b18-04e2-4d67-8ae4-753d060a2dd5 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:08.314003 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-aa893b18-04e2-4d67-8ae4-753d060a2dd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:08.314003 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:08.314003 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:08.314003 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:08.314003 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:08.314003 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:08.314003 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:08.314003 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:08.314003 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:08.314003 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:08.314003 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:08.314003 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:08.314003 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:08.314003 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:08.314003 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:08.315088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:08.315088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:08.315088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:08.315088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:08.315088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:08.315088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:08.315088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:08.315088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:08.315088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:08.315088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:08.315088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:08.315088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:08.315088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:08.315088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:08.315088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:08.315088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:08.315088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:08.316314 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:08.316314 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:08.316314 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:08.316314 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:08.316314 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 290/584] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:08.320628 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa893b18-04e2-4d67-8ae4-753d060a2dd5 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:08.328192 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2a2f4-8afe-4750-8486-76306e107b84 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:08.345570 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:08.354444 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:08.363319 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cc890b-ef89-4de6-8720-9f249a06ebae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:08.364068 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc890b-ef89-4de6-8720-9f249a06ebae None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:08.364427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc890b-ef89-4de6-8720-9f249a06ebae None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:08.364775 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc890b-ef89-4de6-8720-9f249a06ebae None admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:08.365584 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cc890b-ef89-4de6-8720-9f249a06ebae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:08.365966 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cc890b-ef89-4de6-8720-9f249a06ebae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:08.367449 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cc890b-ef89-4de6-8720-9f249a06ebae None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:08.383642 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 295/585] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:08.383642 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cc890b-ef89-4de6-8720-9f249a06ebae None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:08.433952 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa893b18-04e2-4d67-8ae4-753d060a2dd5 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:08.471305 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea528a88-2080-4164-adad-b502f5a2ccde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:08.471912 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea528a88-2080-4164-adad-b502f5a2ccde None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:08.472194 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea528a88-2080-4164-adad-b502f5a2ccde None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:08.472489 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea528a88-2080-4164-adad-b502f5a2ccde None admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:08.473286 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea528a88-2080-4164-adad-b502f5a2ccde None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:08.473594 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea528a88-2080-4164-adad-b502f5a2ccde None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:08.475057 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea528a88-2080-4164-adad-b502f5a2ccde None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:08.485951 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cc890b-ef89-4de6-8720-9f249a06ebae None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:08.494577 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 291/586] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jun 23 01:44:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:08.494577 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea528a88-2080-4164-adad-b502f5a2ccde None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:08.526298 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d5429-5b99-4536-b3f9-806b251aafa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:08.527127 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d5429-5b99-4536-b3f9-806b251aafa6 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:08.527410 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d5429-5b99-4536-b3f9-806b251aafa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:08.527688 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d5429-5b99-4536-b3f9-806b251aafa6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:08.538845 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88d5429-5b99-4536-b3f9-806b251aafa6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:08.540714 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88d5429-5b99-4536-b3f9-806b251aafa6 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:08.559204 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-d88d5429-5b99-4536-b3f9-806b251aafa6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:08.559204 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:08.559204 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:08.559204 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:08.559204 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:08.559204 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:08.559204 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:08.559204 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:08.559204 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:08.559204 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:08.559204 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:08.559204 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:08.559204 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:08.559204 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:08.559204 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:08.560135 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:08.560135 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:08.560135 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:08.560135 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:08.560135 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:08.560135 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:08.560135 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:08.560135 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:08.560135 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:08.560135 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:08.560135 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:08.560135 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:08.560135 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:08.560135 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:08.560135 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:08.560135 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:08.560135 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:08.561003 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:08.561003 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:08.561003 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:08.561003 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:08.561003 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 296/587] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:08.561280 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d5429-5b99-4536-b3f9-806b251aafa6 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:08.594333 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:08.595912 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:08.605210 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea528a88-2080-4164-adad-b502f5a2ccde None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:08.640954 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd46861f-8b6f-4e2a-a270-c8df6080072b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:08.641615 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46861f-8b6f-4e2a-a270-c8df6080072b None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:08.641900 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46861f-8b6f-4e2a-a270-c8df6080072b None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:08.642177 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46861f-8b6f-4e2a-a270-c8df6080072b None admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:08.642908 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd46861f-8b6f-4e2a-a270-c8df6080072b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:08.643235 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd46861f-8b6f-4e2a-a270-c8df6080072b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:08.645088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd46861f-8b6f-4e2a-a270-c8df6080072b None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:08.663157 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 292/588] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:08.663157 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd46861f-8b6f-4e2a-a270-c8df6080072b None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:08.673705 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d5429-5b99-4536-b3f9-806b251aafa6 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:08.707503 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b36fcc-d8ef-4835-99a8-40a6adc6e050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:08.708387 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b36fcc-d8ef-4835-99a8-40a6adc6e050 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:08.708730 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b36fcc-d8ef-4835-99a8-40a6adc6e050 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:08.709065 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b36fcc-d8ef-4835-99a8-40a6adc6e050 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:08.716470 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b36fcc-d8ef-4835-99a8-40a6adc6e050 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:08.717981 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b36fcc-d8ef-4835-99a8-40a6adc6e050 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:08.732077 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-77b36fcc-d8ef-4835-99a8-40a6adc6e050 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 01:44:08.732077 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:08.732077 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:08.732077 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:08.732077 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:08.732077 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:08.732077 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:08.732077 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:08.732077 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:08.732077 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:08.732077 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:08.732077 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:08.732077 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 01:44:08.732077 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 01:44:08.732077 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 01:44:08.732077 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 01:44:08.732979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:08.732979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:08.732979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 01:44:08.732979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:08.732979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 01:44:08.732979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:08.732979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:08.732979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:08.732979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:08.732979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:08.732979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:08.732979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:08.732979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:08.732979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:08.732979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:08.732979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:08.732979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:08.732979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:08.733986 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:08.733986 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:08.733986 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 01:44:08.733986 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 01:44:08.733986 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 01:44:08.733986 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 01:44:08.733986 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 01:44:08.733986 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 01:44:08.733986 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 01:44:08.733986 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:08.733986 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 297/589] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jun 23 01:44:08 2022] GET /identity/v3/users/neutron => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:08.733986 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b36fcc-d8ef-4835-99a8-40a6adc6e050 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:08.771075 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd46861f-8b6f-4e2a-a270-c8df6080072b None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:08.806869 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c3a51-9293-4892-b511-9c05e4d10e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:08.807778 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c3a51-9293-4892-b511-9c05e4d10e57 None admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:08.808055 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c3a51-9293-4892-b511-9c05e4d10e57 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:08.808331 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c3a51-9293-4892-b511-9c05e4d10e57 None admin] PATH_INFO: `/v3/projects/a44a5e0e3e2d4b8e9b9cec7b5b5c725c/users/ae4e83df33b64a3ebf28f54e2d94fe63/roles/c20e054f00164967a4c16e7500c93361` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:08.823969 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8c3a51-9293-4892-b511-9c05e4d10e57 None admin] RBAC: Authorizing `identity:create_grant(project_id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c, user_id=ae4e83df33b64a3ebf28f54e2d94fe63, role_id=c20e054f00164967a4c16e7500c93361)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:08.825436 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8c3a51-9293-4892-b511-9c05e4d10e57 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:08.834892 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:08.841408 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 293/590] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jun 23 01:44:08 2022] PUT /identity/v3/projects/a44a5e0e3e2d4b8e9b9cec7b5b5c725c/users/ae4e83df33b64a3ebf28f54e2d94fe63/roles/c20e054f00164967a4c16e7500c93361 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:44:08.842379 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c3a51-9293-4892-b511-9c05e4d10e57 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:08.845149 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:08.856919 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b36fcc-d8ef-4835-99a8-40a6adc6e050 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:08.893657 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2fd51a-002c-491f-a569-6fdc1e89d97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:08.894676 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2fd51a-002c-491f-a569-6fdc1e89d97d None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:08.895141 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2fd51a-002c-491f-a569-6fdc1e89d97d None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:08.895586 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2fd51a-002c-491f-a569-6fdc1e89d97d None admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:08.896523 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2fd51a-002c-491f-a569-6fdc1e89d97d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:08.896998 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2fd51a-002c-491f-a569-6fdc1e89d97d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:08.898820 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2fd51a-002c-491f-a569-6fdc1e89d97d None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:08.920499 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 298/591] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:08.921806 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2fd51a-002c-491f-a569-6fdc1e89d97d None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:08.981413 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c3a51-9293-4892-b511-9c05e4d10e57 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:09.034001 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ce5ed-9bd3-48fb-bff5-90734564ef65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:09.034001 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ce5ed-9bd3-48fb-bff5-90734564ef65 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:09.034001 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ce5ed-9bd3-48fb-bff5-90734564ef65 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:09.034001 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ce5ed-9bd3-48fb-bff5-90734564ef65 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:09.043475 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2fd51a-002c-491f-a569-6fdc1e89d97d None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:09.044131 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7ce5ed-9bd3-48fb-bff5-90734564ef65 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:09.045701 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7ce5ed-9bd3-48fb-bff5-90734564ef65 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:09.063245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-cb7ce5ed-9bd3-48fb-bff5-90734564ef65 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 01:44:09.063245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:09.063245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:09.063245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:09.063245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:09.063245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:09.063245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:09.063245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:09.063245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:09.063245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:09.063245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:09.063245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:09.063245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 01:44:09.063245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 01:44:09.063245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 01:44:09.063245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 01:44:09.064198 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:09.064198 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:09.064198 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 01:44:09.064198 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:09.064198 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 01:44:09.064198 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:09.064198 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:09.064198 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:09.064198 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:09.064198 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:09.064198 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:09.064198 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:09.064198 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:09.064198 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:09.064198 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:09.064198 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:09.064198 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:09.064198 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:09.065083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:09.065083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:09.065083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 01:44:09.065083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 01:44:09.065083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 01:44:09.065083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 01:44:09.065083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 01:44:09.065083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 01:44:09.065083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 01:44:09.065083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:09.073028 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 294/592] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:44:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:09.073864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ce5ed-9bd3-48fb-bff5-90734564ef65 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:09.085924 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21dbd28-3835-4ac0-a44a-285b78c60969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:09.086649 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21dbd28-3835-4ac0-a44a-285b78c60969 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:09.086878 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21dbd28-3835-4ac0-a44a-285b78c60969 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:09.087152 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21dbd28-3835-4ac0-a44a-285b78c60969 None admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:09.088048 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21dbd28-3835-4ac0-a44a-285b78c60969 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:09.088354 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21dbd28-3835-4ac0-a44a-285b78c60969 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:09.089785 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21dbd28-3835-4ac0-a44a-285b78c60969 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:09.112233 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 299/593] 10.222.0.26 () {56 vars in 1139 bytes} [Thu Jun 23 01:44:08 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:09.113197 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21dbd28-3835-4ac0-a44a-285b78c60969 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:09.124119 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:09.155141 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:09.208449 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ce5ed-9bd3-48fb-bff5-90734564ef65 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:09.235424 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21dbd28-3835-4ac0-a44a-285b78c60969 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:09.246001 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b4d737-1d79-406f-9390-4792c142213d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:09.246983 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b4d737-1d79-406f-9390-4792c142213d None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:09.247419 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b4d737-1d79-406f-9390-4792c142213d None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:09.247818 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b4d737-1d79-406f-9390-4792c142213d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:09.255687 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b4d737-1d79-406f-9390-4792c142213d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:09.257241 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b4d737-1d79-406f-9390-4792c142213d None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:09.275147 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-87b4d737-1d79-406f-9390-4792c142213d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:09.275147 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:09.275147 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:09.275147 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:09.275147 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:09.275147 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:09.275147 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:09.275147 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:09.275147 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:09.275147 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:09.275147 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:09.275147 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:09.275147 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 01:44:09.275147 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 01:44:09.275147 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 01:44:09.276083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 01:44:09.276083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:09.276083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:09.276083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:09.276083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:09.276083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:09.276083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:09.276083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:09.276083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:09.276083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:09.276083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:09.276083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:09.276083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:09.276083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:09.276083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:09.276083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:09.276083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:09.276938 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 01:44:09.276938 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 01:44:09.276938 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 01:44:09.276938 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 01:44:09.276938 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 01:44:09.276938 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 01:44:09.276938 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:09.276938 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:09.279722 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a433140a-f684-400a-85e6-bc7fb6fc9764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:09.280603 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433140a-f684-400a-85e6-bc7fb6fc9764 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:09.281050 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433140a-f684-400a-85e6-bc7fb6fc9764 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:09.281448 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 295/594] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:44:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:09.281808 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433140a-f684-400a-85e6-bc7fb6fc9764 None admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:09.282526 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b4d737-1d79-406f-9390-4792c142213d None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:09.283127 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a433140a-f684-400a-85e6-bc7fb6fc9764 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:09.283594 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a433140a-f684-400a-85e6-bc7fb6fc9764 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:09.285316 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a433140a-f684-400a-85e6-bc7fb6fc9764 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:09.313228 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 300/595] 10.222.0.26 () {56 vars in 1137 bytes} [Thu Jun 23 01:44:09 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:09.314329 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a433140a-f684-400a-85e6-bc7fb6fc9764 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:09.388020 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:09.413367 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b4d737-1d79-406f-9390-4792c142213d None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:09.428864 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a433140a-f684-400a-85e6-bc7fb6fc9764 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:09.441281 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:09.453683 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462f994f-6157-49bf-98e8-0d224b56f234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:09.454493 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f994f-6157-49bf-98e8-0d224b56f234 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:09.454742 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f994f-6157-49bf-98e8-0d224b56f234 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:09.455009 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f994f-6157-49bf-98e8-0d224b56f234 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:09.464206 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f543ff-96aa-4543-b721-1692660922af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:09.464879 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f543ff-96aa-4543-b721-1692660922af None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:09.465162 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f543ff-96aa-4543-b721-1692660922af None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:09.465436 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f543ff-96aa-4543-b721-1692660922af None admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:09.466238 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f543ff-96aa-4543-b721-1692660922af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:09.466547 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f543ff-96aa-4543-b721-1692660922af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:09.466702 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462f994f-6157-49bf-98e8-0d224b56f234 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:09.468020 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f543ff-96aa-4543-b721-1692660922af None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:09.468118 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462f994f-6157-49bf-98e8-0d224b56f234 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:09.483261 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 301/596] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jun 23 01:44:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:09.483261 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f543ff-96aa-4543-b721-1692660922af None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:09.486707 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-462f994f-6157-49bf-98e8-0d224b56f234 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:09.486707 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:09.486707 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:09.486707 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:09.486707 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:09.486707 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:09.486707 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:09.486707 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:09.486707 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:09.486707 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:09.486707 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:09.486707 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:09.486707 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:09.486707 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:09.486707 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:09.487641 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:09.487641 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:09.487641 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:09.487641 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:09.487641 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:09.487641 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:09.487641 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:09.487641 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:09.487641 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:09.487641 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:09.487641 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:09.487641 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:09.487641 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:09.487641 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:09.487641 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:09.487641 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:09.487641 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:09.488771 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:09.488771 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:09.488771 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:09.488771 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:09.491627 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 296/597] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:09.493155 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee80675d-9785-45eb-aaa5-0ae68cc82f44 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:09.493489 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee80675d-9785-45eb-aaa5-0ae68cc82f44 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:09.493765 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee80675d-9785-45eb-aaa5-0ae68cc82f44 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:09.495184 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 297/598] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:09.499807 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee80675d-9785-45eb-aaa5-0ae68cc82f44 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:09.595443 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f543ff-96aa-4543-b721-1692660922af None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:09.609508 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee80675d-9785-45eb-aaa5-0ae68cc82f44 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:09.623821 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:09.633408 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8117ac5a-b59d-45fe-9c46-6a7bc59f18dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:09.634211 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117ac5a-b59d-45fe-9c46-6a7bc59f18dd None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:09.634491 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117ac5a-b59d-45fe-9c46-6a7bc59f18dd None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:09.634764 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117ac5a-b59d-45fe-9c46-6a7bc59f18dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:09.645962 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8117ac5a-b59d-45fe-9c46-6a7bc59f18dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:09.646422 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c0abb-aac0-458c-9c59-79ea2b5c667c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:09.647416 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c0abb-aac0-458c-9c59-79ea2b5c667c None admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:09.647520 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8117ac5a-b59d-45fe-9c46-6a7bc59f18dd None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:09.647796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c0abb-aac0-458c-9c59-79ea2b5c667c None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:09.648127 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c0abb-aac0-458c-9c59-79ea2b5c667c None admin] PATH_INFO: `/v3/projects/a44a5e0e3e2d4b8e9b9cec7b5b5c725c/users/1c081b4cf34146ca8eb197b62a133433/roles/c20e054f00164967a4c16e7500c93361` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:09.663115 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7c0abb-aac0-458c-9c59-79ea2b5c667c None admin] RBAC: Authorizing `identity:create_grant(project_id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c, user_id=1c081b4cf34146ca8eb197b62a133433, role_id=c20e054f00164967a4c16e7500c93361)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:09.664676 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7c0abb-aac0-458c-9c59-79ea2b5c667c None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:09.664945 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-8117ac5a-b59d-45fe-9c46-6a7bc59f18dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:09.664945 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:09.664945 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:09.664945 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:09.664945 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:09.664945 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:09.664945 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:09.664945 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:09.664945 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:09.664945 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:09.664945 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:09.664945 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:09.664945 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:09.664945 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:09.664945 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:09.665966 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:09.665966 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:09.665966 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:09.665966 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:09.665966 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:09.665966 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:09.665966 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:09.665966 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:09.665966 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:09.665966 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:09.665966 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:09.665966 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:09.665966 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:09.665966 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:09.665966 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:09.665966 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:09.665966 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:09.666863 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:09.666863 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:09.666863 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:09.666863 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:09.666863 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 302/599] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:09.667340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8117ac5a-b59d-45fe-9c46-6a7bc59f18dd None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:09.684455 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 298/600] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jun 23 01:44:09 2022] PUT /identity/v3/projects/a44a5e0e3e2d4b8e9b9cec7b5b5c725c/users/1c081b4cf34146ca8eb197b62a133433/roles/c20e054f00164967a4c16e7500c93361 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:44:09.685860 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c752f52-9051-48d4-89fc-c9ed1fd7a134 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:09.686214 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c752f52-9051-48d4-89fc-c9ed1fd7a134 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:09.686553 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c752f52-9051-48d4-89fc-c9ed1fd7a134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:09.704915 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-4c752f52-9051-48d4-89fc-c9ed1fd7a134 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:09.706426 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-4c752f52-9051-48d4-89fc-c9ed1fd7a134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:09.728486 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:09.791275 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 299/601] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:44:09.791275 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c752f52-9051-48d4-89fc-c9ed1fd7a134 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:09.833038 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8117ac5a-b59d-45fe-9c46-6a7bc59f18dd None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:09.875822 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1281e333-7642-4fa0-842e-4f696dc45b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:09.876253 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281e333-7642-4fa0-842e-4f696dc45b5c None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:09.876531 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281e333-7642-4fa0-842e-4f696dc45b5c None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:09.876804 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281e333-7642-4fa0-842e-4f696dc45b5c None admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:09.877532 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1281e333-7642-4fa0-842e-4f696dc45b5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:09.877842 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1281e333-7642-4fa0-842e-4f696dc45b5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:09.879284 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1281e333-7642-4fa0-842e-4f696dc45b5c None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:09.892786 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 303/602] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:09.897771 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1281e333-7642-4fa0-842e-4f696dc45b5c None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:09.923178 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c752f52-9051-48d4-89fc-c9ed1fd7a134 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:09.938392 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:09.971056 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16114aa3-1ee3-4c4d-9b20-17be43853036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:09.971750 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16114aa3-1ee3-4c4d-9b20-17be43853036 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:09.972023 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16114aa3-1ee3-4c4d-9b20-17be43853036 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:09.972327 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16114aa3-1ee3-4c4d-9b20-17be43853036 None admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:09.973061 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16114aa3-1ee3-4c4d-9b20-17be43853036 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:09.973358 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16114aa3-1ee3-4c4d-9b20-17be43853036 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:09.974743 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16114aa3-1ee3-4c4d-9b20-17be43853036 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:09.986912 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 300/603] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:09.988150 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16114aa3-1ee3-4c4d-9b20-17be43853036 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:10.005100 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:10.023352 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1281e333-7642-4fa0-842e-4f696dc45b5c None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:10.070581 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb952c-56d9-4e50-ab1d-9179f106ba5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:10.072150 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb952c-56d9-4e50-ab1d-9179f106ba5f None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:10.072572 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb952c-56d9-4e50-ab1d-9179f106ba5f None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:10.073440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb952c-56d9-4e50-ab1d-9179f106ba5f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:10.087841 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeb952c-56d9-4e50-ab1d-9179f106ba5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:44:10.097710 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16114aa3-1ee3-4c4d-9b20-17be43853036 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:10.098672 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-ebeb952c-56d9-4e50-ab1d-9179f106ba5f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 01:44:10.098672 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:10.098672 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:10.098672 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:10.098672 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:10.098672 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:10.098672 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:10.098672 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:10.098672 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:10.098672 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:10.098672 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:10.098672 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:10.098672 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:44:10.098672 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:44:10.098672 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:44:10.098672 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:44:10.099749 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:44:10.099749 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:44:10.099749 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:44:10.099749 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:44:10.099749 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:10.099749 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:10.099749 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:10.099749 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:10.099749 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:10.099749 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:10.099749 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:10.099749 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:10.099749 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:10.099749 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:10.099749 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:10.099749 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:10.100706 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:10.100706 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:10.100706 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:10.100706 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:10.100706 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:44:10.100706 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:44:10.100706 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:44:10.100706 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:44:10.100706 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:44:10.100706 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:44:10.100706 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 01:44:10.100706 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:10.101354 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 304/604] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jun 23 01:44:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:10.106313 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb952c-56d9-4e50-ab1d-9179f106ba5f None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:10.141988 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6afda89-3789-4f07-99b4-517893a9835e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:10.143126 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6afda89-3789-4f07-99b4-517893a9835e None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:10.143813 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6afda89-3789-4f07-99b4-517893a9835e None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:10.144257 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6afda89-3789-4f07-99b4-517893a9835e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:10.154102 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6afda89-3789-4f07-99b4-517893a9835e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:10.155798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6afda89-3789-4f07-99b4-517893a9835e None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:10.175579 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-b6afda89-3789-4f07-99b4-517893a9835e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:10.175579 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:10.175579 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:10.175579 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:10.175579 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:10.175579 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:10.175579 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:10.175579 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:10.175579 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:10.175579 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:10.175579 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:10.175579 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:10.175579 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 01:44:10.175579 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 01:44:10.175579 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 01:44:10.176662 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 01:44:10.176662 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:10.176662 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:10.176662 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:10.176662 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:10.176662 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:10.176662 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:10.176662 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:10.176662 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:10.176662 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:10.176662 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:10.176662 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:10.176662 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:10.176662 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:10.176662 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:10.176662 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:10.176662 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:10.177725 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 01:44:10.177725 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 01:44:10.177725 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 01:44:10.177725 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 01:44:10.177725 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 01:44:10.177725 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 01:44:10.177725 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:10.177725 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:10.180905 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 301/605] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:44:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:10.181819 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6afda89-3789-4f07-99b4-517893a9835e None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:10.236114 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:10.242570 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb952c-56d9-4e50-ab1d-9179f106ba5f None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:10.267526 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:10.289324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647c4398-dad2-4881-8619-b85a27f5c81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:10.290614 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c4398-dad2-4881-8619-b85a27f5c81c None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:10.291452 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c4398-dad2-4881-8619-b85a27f5c81c None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:10.291887 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c4398-dad2-4881-8619-b85a27f5c81c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:10.299500 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6afda89-3789-4f07-99b4-517893a9835e None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:10.301255 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647c4398-dad2-4881-8619-b85a27f5c81c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:10.303833 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647c4398-dad2-4881-8619-b85a27f5c81c None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:10.323547 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-647c4398-dad2-4881-8619-b85a27f5c81c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:10.323547 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:10.323547 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:10.323547 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:10.323547 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:10.323547 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:10.323547 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:10.323547 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:10.323547 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:10.323547 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:10.323547 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:10.323547 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:10.323547 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 01:44:10.323547 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 01:44:10.323547 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 01:44:10.324643 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 01:44:10.324643 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:10.324643 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:10.324643 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:10.324643 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:10.324643 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:10.324643 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:10.324643 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:10.324643 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:10.324643 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:10.324643 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:10.324643 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:10.324643 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:10.324643 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:10.324643 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:10.324643 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:10.324643 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:10.325720 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 01:44:10.325720 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 01:44:10.325720 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 01:44:10.325720 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 01:44:10.325720 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 01:44:10.325720 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 01:44:10.325720 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:10.325720 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:10.328529 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 305/606] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:44:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:10.330036 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647c4398-dad2-4881-8619-b85a27f5c81c None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:10.337848 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f12c0e-14c1-4eff-a997-202d383a2586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:10.338630 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f12c0e-14c1-4eff-a997-202d383a2586 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:10.339001 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f12c0e-14c1-4eff-a997-202d383a2586 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:10.339409 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f12c0e-14c1-4eff-a997-202d383a2586 None admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:10.340435 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f12c0e-14c1-4eff-a997-202d383a2586 None admin] RBAC: Adding query filter params (name=service) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:10.340830 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f12c0e-14c1-4eff-a997-202d383a2586 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:10.342310 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f12c0e-14c1-4eff-a997-202d383a2586 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:10.351381 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 302/607] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Jun 23 01:44:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:10.352292 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f12c0e-14c1-4eff-a997-202d383a2586 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:10.446588 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647c4398-dad2-4881-8619-b85a27f5c81c None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:10.474665 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f12c0e-14c1-4eff-a997-202d383a2586 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:10.482007 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff1eca2-bf80-42e7-a3e5-d5f651a4fbe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:10.482778 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff1eca2-bf80-42e7-a3e5-d5f651a4fbe3 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:10.483153 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff1eca2-bf80-42e7-a3e5-d5f651a4fbe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:10.483549 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff1eca2-bf80-42e7-a3e5-d5f651a4fbe3 None admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:10.484489 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff1eca2-bf80-42e7-a3e5-d5f651a4fbe3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:10.484841 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff1eca2-bf80-42e7-a3e5-d5f651a4fbe3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:10.486368 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff1eca2-bf80-42e7-a3e5-d5f651a4fbe3 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:10.503787 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 306/608] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jun 23 01:44:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:10.504834 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff1eca2-bf80-42e7-a3e5-d5f651a4fbe3 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:10.517071 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a2c8f-0577-46d1-8db7-9dd196bfe95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:10.518791 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a2c8f-0577-46d1-8db7-9dd196bfe95e None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:10.519325 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a2c8f-0577-46d1-8db7-9dd196bfe95e None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:10.519755 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a2c8f-0577-46d1-8db7-9dd196bfe95e None admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:10.521084 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79a2c8f-0577-46d1-8db7-9dd196bfe95e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:10.521565 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79a2c8f-0577-46d1-8db7-9dd196bfe95e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:10.524589 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79a2c8f-0577-46d1-8db7-9dd196bfe95e None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:10.528783 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:10.532648 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:10.544179 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 303/609] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jun 23 01:44:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:10.545697 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a2c8f-0577-46d1-8db7-9dd196bfe95e None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:10.636654 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff1eca2-bf80-42e7-a3e5-d5f651a4fbe3 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:10.651839 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a2c8f-0577-46d1-8db7-9dd196bfe95e None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:10.677709 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83df24f-9721-420f-9cb3-0f6749ba1742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:10.678676 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83df24f-9721-420f-9cb3-0f6749ba1742 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:10.679044 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83df24f-9721-420f-9cb3-0f6749ba1742 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:10.679446 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83df24f-9721-420f-9cb3-0f6749ba1742 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:10.691322 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83df24f-9721-420f-9cb3-0f6749ba1742 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:10.692881 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83df24f-9721-420f-9cb3-0f6749ba1742 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:10.695094 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a2f656-150e-41cb-9abf-e84d22d78168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:10.696555 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a2f656-150e-41cb-9abf-e84d22d78168 None admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:10.697054 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a2f656-150e-41cb-9abf-e84d22d78168 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:10.697479 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a2f656-150e-41cb-9abf-e84d22d78168 None admin] PATH_INFO: `/v3/projects/a44a5e0e3e2d4b8e9b9cec7b5b5c725c/users/948e4905de44473bb7a45e1b521cf095/roles/c20e054f00164967a4c16e7500c93361` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:10.711289 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-b83df24f-9721-420f-9cb3-0f6749ba1742 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:10.711289 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:10.711289 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:10.711289 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:10.711289 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:10.711289 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:10.711289 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:10.711289 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:10.711289 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:10.711289 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:10.711289 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:10.711289 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:10.711289 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:10.711289 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:10.711289 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:10.712366 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:10.712366 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:10.712366 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:10.712366 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:10.712366 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:10.712366 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:10.712366 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:10.712366 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:10.712366 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:10.712366 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:10.712366 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:10.712366 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:10.712366 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:10.712366 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:10.712366 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:10.712366 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:10.712366 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:10.713428 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:10.713428 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:10.713428 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:10.713428 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:10.714820 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 307/610] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:10.715801 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83df24f-9721-420f-9cb3-0f6749ba1742 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:10.716735 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a2f656-150e-41cb-9abf-e84d22d78168 None admin] RBAC: Authorizing `identity:create_grant(project_id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c, user_id=948e4905de44473bb7a45e1b521cf095, role_id=c20e054f00164967a4c16e7500c93361)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:10.718333 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a2f656-150e-41cb-9abf-e84d22d78168 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:10.738474 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 304/611] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jun 23 01:44:10 2022] PUT /identity/v3/projects/a44a5e0e3e2d4b8e9b9cec7b5b5c725c/users/948e4905de44473bb7a45e1b521cf095/roles/c20e054f00164967a4c16e7500c93361 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:44:10.739596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a2f656-150e-41cb-9abf-e84d22d78168 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:10.798248 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:10.869988 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a2f656-150e-41cb-9abf-e84d22d78168 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:10.878751 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:10.910581 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db6499d-9f50-4bf0-a072-4c39ffc34f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:10.911542 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6499d-9f50-4bf0-a072-4c39ffc34f85 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:10.912083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6499d-9f50-4bf0-a072-4c39ffc34f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:10.912551 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6499d-9f50-4bf0-a072-4c39ffc34f85 None admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:10.913512 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db6499d-9f50-4bf0-a072-4c39ffc34f85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:10.913970 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db6499d-9f50-4bf0-a072-4c39ffc34f85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:10.915648 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db6499d-9f50-4bf0-a072-4c39ffc34f85 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:10.929125 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 305/612] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:10.934621 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db6499d-9f50-4bf0-a072-4c39ffc34f85 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:10.951694 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83df24f-9721-420f-9cb3-0f6749ba1742 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:10.999195 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f77007c-9884-4397-85cb-7af42c0e3c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:11.000274 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77007c-9884-4397-85cb-7af42c0e3c93 None admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:11.000551 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77007c-9884-4397-85cb-7af42c0e3c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:11.000850 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77007c-9884-4397-85cb-7af42c0e3c93 None admin] PATH_INFO: `/v3/projects/a44a5e0e3e2d4b8e9b9cec7b5b5c725c/users/dc5bb02f8eb64ac980e3218470b3c1b0/roles/c20e054f00164967a4c16e7500c93361` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:11.020964 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f77007c-9884-4397-85cb-7af42c0e3c93 None admin] RBAC: Authorizing `identity:create_grant(project_id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c, user_id=dc5bb02f8eb64ac980e3218470b3c1b0, role_id=c20e054f00164967a4c16e7500c93361)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:11.022519 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f77007c-9884-4397-85cb-7af42c0e3c93 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:11.040037 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 308/613] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jun 23 01:44:10 2022] PUT /identity/v3/projects/a44a5e0e3e2d4b8e9b9cec7b5b5c725c/users/dc5bb02f8eb64ac980e3218470b3c1b0/roles/c20e054f00164967a4c16e7500c93361 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:44:11.041494 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff363f36-c395-4bb5-8418-cb4f8d652df1 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:11.041841 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff363f36-c395-4bb5-8418-cb4f8d652df1 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:11.042185 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff363f36-c395-4bb5-8418-cb4f8d652df1 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:11.044800 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 309/614] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:11.052540 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82180017-1aa2-4ffc-8a04-3384139c5dbb None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:11.052846 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82180017-1aa2-4ffc-8a04-3384139c5dbb None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:11.053090 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82180017-1aa2-4ffc-8a04-3384139c5dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:11.064244 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db6499d-9f50-4bf0-a072-4c39ffc34f85 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:11.075940 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:11.076252 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-82180017-1aa2-4ffc-8a04-3384139c5dbb None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:11.077658 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-82180017-1aa2-4ffc-8a04-3384139c5dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:11.118091 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3530395-0b16-4951-a4ff-a7865e1d8173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:11.119322 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3530395-0b16-4951-a4ff-a7865e1d8173 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:11.119808 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3530395-0b16-4951-a4ff-a7865e1d8173 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:11.120277 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3530395-0b16-4951-a4ff-a7865e1d8173 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:11.131310 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3530395-0b16-4951-a4ff-a7865e1d8173 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:11.133278 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3530395-0b16-4951-a4ff-a7865e1d8173 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:11.153870 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-a3530395-0b16-4951-a4ff-a7865e1d8173 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 01:44:11.153870 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:11.153870 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:11.153870 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:11.153870 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:11.153870 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:11.153870 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:11.153870 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:11.153870 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:11.153870 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:11.153870 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:11.153870 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:11.153870 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 01:44:11.153870 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 01:44:11.153870 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 01:44:11.153870 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 01:44:11.154864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:11.154864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:11.154864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 01:44:11.154864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:11.154864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 01:44:11.154864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:11.154864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:11.154864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:11.154864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:11.154864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:11.154864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:11.154864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:11.154864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:11.154864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:11.154864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:11.154864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:11.154864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:11.154864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:11.155838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:11.155838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:11.155838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 01:44:11.155838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 01:44:11.155838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 01:44:11.155838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 01:44:11.155838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 01:44:11.155838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 01:44:11.155838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 01:44:11.155838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:11.163471 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 306/615] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jun 23 01:44:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:11.165933 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3530395-0b16-4951-a4ff-a7865e1d8173 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:11.198561 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 310/616] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:44:11.252084 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82180017-1aa2-4ffc-8a04-3384139c5dbb None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:11.315658 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3530395-0b16-4951-a4ff-a7865e1d8173 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:11.327388 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:11.380642 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ee41d5-f38d-4b47-b277-a38e5157e26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:11.381309 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ee41d5-f38d-4b47-b277-a38e5157e26a None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:11.381584 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ee41d5-f38d-4b47-b277-a38e5157e26a None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:11.381868 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ee41d5-f38d-4b47-b277-a38e5157e26a None admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:11.382692 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ee41d5-f38d-4b47-b277-a38e5157e26a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:11.382994 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ee41d5-f38d-4b47-b277-a38e5157e26a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:11.384468 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ee41d5-f38d-4b47-b277-a38e5157e26a None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:11.417342 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 307/617] 10.222.0.26 () {56 vars in 1135 bytes} [Thu Jun 23 01:44:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:11.420489 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82180017-1aa2-4ffc-8a04-3384139c5dbb None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:11.421768 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ee41d5-f38d-4b47-b277-a38e5157e26a None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:11.459593 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:11.518706 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef770a-b4c6-4faf-bf96-e98579369ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:11.519524 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef770a-b4c6-4faf-bf96-e98579369ca6 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:11.519799 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef770a-b4c6-4faf-bf96-e98579369ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:11.520069 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef770a-b4c6-4faf-bf96-e98579369ca6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:11.539154 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ef770a-b4c6-4faf-bf96-e98579369ca6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:44:11.547618 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-a6ef770a-b4c6-4faf-bf96-e98579369ca6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 01:44:11.547618 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:11.547618 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:11.547618 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:11.547618 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:11.547618 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:11.547618 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:11.547618 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:11.547618 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:11.547618 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:11.547618 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:11.547618 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:11.547618 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:44:11.547618 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:44:11.547618 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:44:11.547618 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:44:11.548543 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:44:11.548543 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:44:11.548543 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:44:11.548543 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:44:11.548543 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:11.548543 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:11.548543 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:11.548543 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:11.548543 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:11.548543 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:11.548543 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:11.548543 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:11.548543 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:11.548543 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:11.548543 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:11.548543 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:11.549392 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:11.549392 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:11.549392 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:11.549392 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:11.549392 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:44:11.549392 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:44:11.549392 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:44:11.549392 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:44:11.549392 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:44:11.549392 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:44:11.549392 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 01:44:11.549392 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:11.550045 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 311/618] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jun 23 01:44:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:44:11.557562 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef770a-b4c6-4faf-bf96-e98579369ca6 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:11.570970 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ee41d5-f38d-4b47-b277-a38e5157e26a None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:11.655941 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d35f40-315a-4ddd-8109-40ace6f6f893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:11.656743 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d35f40-315a-4ddd-8109-40ace6f6f893 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:11.657022 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d35f40-315a-4ddd-8109-40ace6f6f893 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:11.657294 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d35f40-315a-4ddd-8109-40ace6f6f893 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:11.681609 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef770a-b4c6-4faf-bf96-e98579369ca6 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:11.682972 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d35f40-315a-4ddd-8109-40ace6f6f893 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:11.684703 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d35f40-315a-4ddd-8109-40ace6f6f893 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:11.703574 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:11.705785 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-64d35f40-315a-4ddd-8109-40ace6f6f893 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:11.705785 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:11.705785 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:11.705785 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:11.705785 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:11.705785 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:11.705785 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:11.705785 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:11.705785 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:11.705785 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:11.705785 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:11.705785 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:11.705785 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:11.705785 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:11.705785 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:11.706657 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:11.706657 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:11.706657 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:11.706657 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:11.706657 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:11.706657 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:11.706657 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:11.706657 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:11.706657 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:11.706657 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:11.706657 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:11.706657 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:11.706657 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:11.706657 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:11.706657 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:11.706657 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:11.706657 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:11.707711 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:11.707711 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:11.707711 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:11.707711 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:11.710072 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 308/619] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:11.715738 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d35f40-315a-4ddd-8109-40ace6f6f893 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:11.757687 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3b01f9-3a7f-4c8d-a837-8a7d1ac8b1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:11.758725 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b01f9-3a7f-4c8d-a837-8a7d1ac8b1a7 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:11.759260 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b01f9-3a7f-4c8d-a837-8a7d1ac8b1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:11.760756 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b01f9-3a7f-4c8d-a837-8a7d1ac8b1a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:11.761717 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3b01f9-3a7f-4c8d-a837-8a7d1ac8b1a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:11.763675 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3b01f9-3a7f-4c8d-a837-8a7d1ac8b1a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:11.766388 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3b01f9-3a7f-4c8d-a837-8a7d1ac8b1a7 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:11.794426 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 312/620] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Jun 23 01:44:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:11.795731 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3b01f9-3a7f-4c8d-a837-8a7d1ac8b1a7 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:11.871628 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:11.909061 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d35f40-315a-4ddd-8109-40ace6f6f893 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:11.956783 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2035e8be-8174-453e-bf02-b525224028f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:11.957450 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035e8be-8174-453e-bf02-b525224028f0 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:11.957727 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035e8be-8174-453e-bf02-b525224028f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:11.957995 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035e8be-8174-453e-bf02-b525224028f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:11.958718 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2035e8be-8174-453e-bf02-b525224028f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:11.959014 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2035e8be-8174-453e-bf02-b525224028f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:11.960430 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2035e8be-8174-453e-bf02-b525224028f0 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:11.972858 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 309/621] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:11.974193 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9568a9-5ce1-4ba6-96e6-87cb86828bca None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:11.974477 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9568a9-5ce1-4ba6-96e6-87cb86828bca None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:11.974747 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9568a9-5ce1-4ba6-96e6-87cb86828bca None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:11.976456 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 310/622] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:11.978304 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9568a9-5ce1-4ba6-96e6-87cb86828bca None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:11.980754 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3b01f9-3a7f-4c8d-a837-8a7d1ac8b1a7 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:12.025248 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec341e6-e677-4653-abb0-af14aab851bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:12.026045 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec341e6-e677-4653-abb0-af14aab851bc None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:12.026579 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec341e6-e677-4653-abb0-af14aab851bc None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:12.026854 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec341e6-e677-4653-abb0-af14aab851bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:12.043223 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec341e6-e677-4653-abb0-af14aab851bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:12.044740 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec341e6-e677-4653-abb0-af14aab851bc None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:12.069499 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-6ec341e6-e677-4653-abb0-af14aab851bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:12.069499 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:12.069499 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:12.069499 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:12.069499 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:12.069499 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:12.069499 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:12.069499 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:12.069499 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:12.069499 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:12.069499 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:12.069499 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:12.069499 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:12.069499 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:12.069499 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:12.070603 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:12.070603 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:12.070603 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:12.070603 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:12.070603 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:12.070603 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:12.070603 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:12.070603 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:12.070603 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:12.070603 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:12.070603 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:12.070603 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:12.070603 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:12.070603 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:12.070603 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:12.070603 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:12.070603 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:12.071725 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:12.071725 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:12.071725 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:12.071725 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:12.079174 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 313/623] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:12.081430 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b599c2f6-0583-4447-b1ce-d28e8e1220e9 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:12.081921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b599c2f6-0583-4447-b1ce-d28e8e1220e9 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:12.082342 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b599c2f6-0583-4447-b1ce-d28e8e1220e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:12.103152 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-b599c2f6-0583-4447-b1ce-d28e8e1220e9 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:12.104835 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-b599c2f6-0583-4447-b1ce-d28e8e1220e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:12.108616 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9568a9-5ce1-4ba6-96e6-87cb86828bca None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:12.126563 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:12.146487 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828c66d-94a7-45d1-88b5-ea3d767d97f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:12.147840 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828c66d-94a7-45d1-88b5-ea3d767d97f3 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:12.147840 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828c66d-94a7-45d1-88b5-ea3d767d97f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:12.148166 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828c66d-94a7-45d1-88b5-ea3d767d97f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:12.151002 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:12.156174 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1828c66d-94a7-45d1-88b5-ea3d767d97f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:12.158632 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1828c66d-94a7-45d1-88b5-ea3d767d97f3 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:12.173220 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-1828c66d-94a7-45d1-88b5-ea3d767d97f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:12.173220 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:12.173220 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:12.173220 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:12.173220 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:12.173220 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:12.173220 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:12.173220 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:12.173220 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:12.173220 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:12.173220 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:12.173220 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:12.173220 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 01:44:12.173220 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 01:44:12.173220 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 01:44:12.174320 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 01:44:12.174320 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:12.174320 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:12.174320 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:12.174320 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:12.174320 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:12.174320 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:12.174320 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:12.174320 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:12.174320 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:12.174320 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:12.174320 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:12.174320 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:12.174320 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:12.174320 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:12.174320 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:12.174320 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:12.175563 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 01:44:12.175563 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 01:44:12.175563 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 01:44:12.175563 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 01:44:12.175563 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 01:44:12.175563 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 01:44:12.175563 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:12.175563 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:12.182266 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 311/624] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:44:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:12.182266 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828c66d-94a7-45d1-88b5-ea3d767d97f3 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:12.196249 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 314/625] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:44:12.197520 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b599c2f6-0583-4447-b1ce-d28e8e1220e9 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:12.311001 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828c66d-94a7-45d1-88b5-ea3d767d97f3 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:12.321497 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b599c2f6-0583-4447-b1ce-d28e8e1220e9 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:12.355260 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41cc6d5-3140-4232-8911-3c30dfc41798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:12.355260 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41cc6d5-3140-4232-8911-3c30dfc41798 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:12.355260 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41cc6d5-3140-4232-8911-3c30dfc41798 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:12.355260 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41cc6d5-3140-4232-8911-3c30dfc41798 None admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:12.355741 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41cc6d5-3140-4232-8911-3c30dfc41798 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:12.355741 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41cc6d5-3140-4232-8911-3c30dfc41798 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:12.355741 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41cc6d5-3140-4232-8911-3c30dfc41798 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:12.358215 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196b8ac7-f078-423d-bb0d-cacd6de854ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:12.358988 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b8ac7-f078-423d-bb0d-cacd6de854ff None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:12.359400 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b8ac7-f078-423d-bb0d-cacd6de854ff None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:12.359774 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b8ac7-f078-423d-bb0d-cacd6de854ff None admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:12.360675 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196b8ac7-f078-423d-bb0d-cacd6de854ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:12.361080 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196b8ac7-f078-423d-bb0d-cacd6de854ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:12.362669 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196b8ac7-f078-423d-bb0d-cacd6de854ff None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:12.367508 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 312/626] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:12.368353 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41cc6d5-3140-4232-8911-3c30dfc41798 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:12.378763 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 315/627] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jun 23 01:44:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:12.379636 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196b8ac7-f078-423d-bb0d-cacd6de854ff None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:12.396235 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:12.432615 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:12.487395 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41cc6d5-3140-4232-8911-3c30dfc41798 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:12.491968 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196b8ac7-f078-423d-bb0d-cacd6de854ff None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:12.532604 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d54d2-2d4e-4842-b106-3eb40df8b210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:12.533657 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d54d2-2d4e-4842-b106-3eb40df8b210 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:12.534115 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d54d2-2d4e-4842-b106-3eb40df8b210 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:12.534527 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d54d2-2d4e-4842-b106-3eb40df8b210 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:12.543028 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd9e171-624e-4b49-a6af-8e68fcd75724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:12.543964 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267d54d2-2d4e-4842-b106-3eb40df8b210 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:12.544137 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9e171-624e-4b49-a6af-8e68fcd75724 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:12.544492 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9e171-624e-4b49-a6af-8e68fcd75724 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:12.544861 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9e171-624e-4b49-a6af-8e68fcd75724 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:12.546585 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267d54d2-2d4e-4842-b106-3eb40df8b210 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:12.557285 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd9e171-624e-4b49-a6af-8e68fcd75724 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:44:12.562724 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-267d54d2-2d4e-4842-b106-3eb40df8b210 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 01:44:12.562724 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:12.562724 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:12.562724 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:12.562724 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:12.562724 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:12.562724 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:12.562724 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:12.562724 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:12.562724 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:12.562724 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:12.562724 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:12.562724 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 01:44:12.562724 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 01:44:12.562724 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 01:44:12.562724 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 01:44:12.563884 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:12.563884 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:12.563884 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 01:44:12.563884 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:12.563884 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 01:44:12.563884 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:12.563884 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:12.563884 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:12.563884 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:12.563884 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:12.563884 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:12.563884 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:12.563884 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:12.563884 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:12.563884 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:12.563884 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:12.563884 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:12.563884 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:12.565116 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:12.565116 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:12.565116 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 01:44:12.565116 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 01:44:12.565116 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 01:44:12.565116 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 01:44:12.565116 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 01:44:12.565116 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 01:44:12.565116 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 01:44:12.565116 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:12.571550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-6bd9e171-624e-4b49-a6af-8e68fcd75724 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 01:44:12.571550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:12.571550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:12.571550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:12.571550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:12.571550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:12.571550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:12.571550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:12.571550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:12.571550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:12.571550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:12.571550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:12.571550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:44:12.571550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:44:12.571550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:44:12.571550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:44:12.572477 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:44:12.572477 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:44:12.572477 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:44:12.572477 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:44:12.572477 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:12.572477 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:12.572477 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:12.572477 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:12.572477 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:12.572477 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:12.572477 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:12.572477 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:12.572477 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:12.572477 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:12.572477 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:12.572477 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:12.573299 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:12.573299 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:12.573299 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:12.573299 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:12.573299 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:44:12.573299 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:44:12.573299 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:44:12.573299 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:44:12.573299 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:44:12.573299 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:44:12.573299 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 01:44:12.573299 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:12.575481 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 313/628] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jun 23 01:44:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:12.575481 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd9e171-624e-4b49-a6af-8e68fcd75724 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:12.576005 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 316/629] 10.222.0.26 () {56 vars in 1093 bytes} [Thu Jun 23 01:44:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:12.577050 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d54d2-2d4e-4842-b106-3eb40df8b210 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:12.658420 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:12.685713 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d54d2-2d4e-4842-b106-3eb40df8b210 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:12.696996 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd9e171-624e-4b49-a6af-8e68fcd75724 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:12.713533 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:12.722351 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe45ab8-2e76-4867-b5f4-1479819dfd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:12.723171 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe45ab8-2e76-4867-b5f4-1479819dfd75 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:12.723648 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe45ab8-2e76-4867-b5f4-1479819dfd75 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:12.723979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe45ab8-2e76-4867-b5f4-1479819dfd75 None admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:12.724914 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe45ab8-2e76-4867-b5f4-1479819dfd75 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:12.725394 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe45ab8-2e76-4867-b5f4-1479819dfd75 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:12.726876 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe45ab8-2e76-4867-b5f4-1479819dfd75 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:12.734136 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0430405e-b4fb-4dc5-902a-3831bdd0deaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:12.735272 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0430405e-b4fb-4dc5-902a-3831bdd0deaf None admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:12.735722 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0430405e-b4fb-4dc5-902a-3831bdd0deaf None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:12.736182 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0430405e-b4fb-4dc5-902a-3831bdd0deaf None admin] PATH_INFO: `/v3/projects/a44a5e0e3e2d4b8e9b9cec7b5b5c725c/users/28e59688485241c4b6d51f99bbb3d059/roles/c20e054f00164967a4c16e7500c93361` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:12.745506 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 317/630] 10.222.0.26 () {56 vars in 1133 bytes} [Thu Jun 23 01:44:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:12.750476 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe45ab8-2e76-4867-b5f4-1479819dfd75 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:12.752451 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0430405e-b4fb-4dc5-902a-3831bdd0deaf None admin] RBAC: Authorizing `identity:create_grant(project_id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c, user_id=28e59688485241c4b6d51f99bbb3d059, role_id=c20e054f00164967a4c16e7500c93361)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:12.754173 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0430405e-b4fb-4dc5-902a-3831bdd0deaf None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:12.771019 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 314/631] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jun 23 01:44:12 2022] PUT /identity/v3/projects/a44a5e0e3e2d4b8e9b9cec7b5b5c725c/users/28e59688485241c4b6d51f99bbb3d059/roles/c20e054f00164967a4c16e7500c93361 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:44:12.772181 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0430405e-b4fb-4dc5-902a-3831bdd0deaf None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:12.894006 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe45ab8-2e76-4867-b5f4-1479819dfd75 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:12.898523 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0430405e-b4fb-4dc5-902a-3831bdd0deaf None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:12.926677 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:12.948040 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d9051e-b872-4ed9-97ea-de615d715e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:12.949385 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9051e-b872-4ed9-97ea-de615d715e8b None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:12.949919 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9051e-b872-4ed9-97ea-de615d715e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:12.950689 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9051e-b872-4ed9-97ea-de615d715e8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:12.950956 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475354ee-9099-4c95-b252-bb35132b3a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:12.951676 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475354ee-9099-4c95-b252-bb35132b3a8b None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:12.951961 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475354ee-9099-4c95-b252-bb35132b3a8b None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:12.952239 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475354ee-9099-4c95-b252-bb35132b3a8b None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:12.953017 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475354ee-9099-4c95-b252-bb35132b3a8b None admin] RBAC: Adding query filter params (name=service) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:12.953327 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475354ee-9099-4c95-b252-bb35132b3a8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:12.954744 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475354ee-9099-4c95-b252-bb35132b3a8b None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:12.964705 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 318/632] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Jun 23 01:44:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:12.966083 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2badb-2461-418c-80a1-a6fb671ca60c None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:12.966373 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2badb-2461-418c-80a1-a6fb671ca60c None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:12.966656 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2badb-2461-418c-80a1-a6fb671ca60c None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:12.968323 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 319/633] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:12.980269 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a2badb-2461-418c-80a1-a6fb671ca60c None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:12.994224 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:13.037943 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d9051e-b872-4ed9-97ea-de615d715e8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:13.039905 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d9051e-b872-4ed9-97ea-de615d715e8b None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:13.116367 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a2badb-2461-418c-80a1-a6fb671ca60c None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:13.163032 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb71c26c-e992-4bd3-b4f4-0b89946cc740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:13.164206 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb71c26c-e992-4bd3-b4f4-0b89946cc740 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:13.164478 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 315/634] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:44:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 01:44:13.165250 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb71c26c-e992-4bd3-b4f4-0b89946cc740 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:13.165740 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb71c26c-e992-4bd3-b4f4-0b89946cc740 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:13.165937 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233f559-f35c-40b3-8f63-45b0e8429d17 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:13.166220 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233f559-f35c-40b3-8f63-45b0e8429d17 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:13.166495 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233f559-f35c-40b3-8f63-45b0e8429d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:13.180262 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb71c26c-e992-4bd3-b4f4-0b89946cc740 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:13.182319 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb71c26c-e992-4bd3-b4f4-0b89946cc740 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:13.184755 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-5233f559-f35c-40b3-8f63-45b0e8429d17 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:13.186016 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-5233f559-f35c-40b3-8f63-45b0e8429d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:13.205273 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-fb71c26c-e992-4bd3-b4f4-0b89946cc740 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:13.205273 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:13.205273 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:13.205273 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:13.205273 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:13.205273 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:13.205273 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:13.205273 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:13.205273 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:13.205273 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:13.205273 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:13.205273 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:13.205273 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:13.205273 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:13.205273 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:13.206240 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:13.206240 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:13.206240 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:13.206240 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:13.206240 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:13.206240 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:13.206240 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:13.206240 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:13.206240 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:13.206240 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:13.206240 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:13.206240 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:13.206240 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:13.206240 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:13.206240 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:13.206240 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:13.206240 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:13.207227 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:13.207227 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:13.207227 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:13.207227 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:13.209296 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 320/635] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:13.214722 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb71c26c-e992-4bd3-b4f4-0b89946cc740 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:13.255279 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:13.269812 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 316/636] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:44:13.270575 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5233f559-f35c-40b3-8f63-45b0e8429d17 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:13.274189 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:13.334558 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb71c26c-e992-4bd3-b4f4-0b89946cc740 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:13.373185 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64df6148-2b8e-443c-93d6-7fab313c796b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:13.374514 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df6148-2b8e-443c-93d6-7fab313c796b None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:13.375006 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df6148-2b8e-443c-93d6-7fab313c796b None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:13.377444 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df6148-2b8e-443c-93d6-7fab313c796b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:13.389330 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64df6148-2b8e-443c-93d6-7fab313c796b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:13.389495 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5233f559-f35c-40b3-8f63-45b0e8429d17 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:13.391378 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64df6148-2b8e-443c-93d6-7fab313c796b None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:13.410277 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-64df6148-2b8e-443c-93d6-7fab313c796b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:13.410277 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:13.410277 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:13.410277 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:13.410277 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:13.410277 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:13.410277 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:13.410277 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:13.410277 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:13.410277 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:13.410277 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:13.410277 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:13.410277 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:13.410277 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:13.410277 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:13.411403 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:13.411403 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:13.411403 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:13.411403 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:13.411403 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:13.411403 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:13.411403 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:13.411403 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:13.411403 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:13.411403 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:13.411403 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:13.411403 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:13.411403 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:13.411403 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:13.411403 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:13.411403 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:13.411403 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:13.412289 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:13.412289 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:13.412289 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:13.412289 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:13.413130 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 321/637] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:13.413858 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64df6148-2b8e-443c-93d6-7fab313c796b None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:13.426187 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c337db-0ceb-4cd1-969b-f72416872c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:13.427189 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c337db-0ceb-4cd1-969b-f72416872c96 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:13.427678 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c337db-0ceb-4cd1-969b-f72416872c96 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:13.428137 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c337db-0ceb-4cd1-969b-f72416872c96 None admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:13.429046 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c337db-0ceb-4cd1-969b-f72416872c96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:13.429425 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c337db-0ceb-4cd1-969b-f72416872c96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:13.431148 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c337db-0ceb-4cd1-969b-f72416872c96 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:13.442799 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 317/638] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:13.448413 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077c0b4-b19e-4007-aab3-0f565ab625bf None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:13.448702 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077c0b4-b19e-4007-aab3-0f565ab625bf None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:13.448980 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077c0b4-b19e-4007-aab3-0f565ab625bf None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:13.450451 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 318/639] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:13.451154 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077c0b4-b19e-4007-aab3-0f565ab625bf None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:13.511238 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:13.519279 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:13.523431 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64df6148-2b8e-443c-93d6-7fab313c796b None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:13.559911 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077c0b4-b19e-4007-aab3-0f565ab625bf None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:13.561337 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54c8bc-831b-4764-b332-e5c3e947e702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:13.562140 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54c8bc-831b-4764-b332-e5c3e947e702 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:13.563033 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54c8bc-831b-4764-b332-e5c3e947e702 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:13.563033 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54c8bc-831b-4764-b332-e5c3e947e702 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:13.573638 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e54c8bc-831b-4764-b332-e5c3e947e702 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:44:13.581375 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-3e54c8bc-831b-4764-b332-e5c3e947e702 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 01:44:13.581375 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:13.581375 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:13.581375 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:13.581375 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:13.581375 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:13.581375 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:13.581375 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:13.581375 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:13.581375 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:13.581375 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:13.581375 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:13.581375 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:44:13.581375 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:44:13.581375 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:44:13.581375 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:44:13.582403 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:44:13.582403 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:44:13.582403 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:44:13.582403 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:44:13.582403 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:13.582403 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:13.582403 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:13.582403 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:13.582403 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:13.582403 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:13.582403 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:13.582403 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:13.582403 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:13.582403 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:13.582403 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:13.582403 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:13.583693 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:13.583693 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:13.583693 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:13.583693 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:13.583693 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:44:13.583693 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:44:13.583693 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:44:13.583693 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:44:13.583693 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:44:13.583693 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:44:13.583693 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 01:44:13.583693 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:13.583693 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 322/640] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jun 23 01:44:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:13.583693 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54c8bc-831b-4764-b332-e5c3e947e702 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:13.600570 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45de61-01e7-46d4-b327-12d965dd47fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:13.601234 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45de61-01e7-46d4-b327-12d965dd47fe None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:13.601511 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45de61-01e7-46d4-b327-12d965dd47fe None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:13.601776 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45de61-01e7-46d4-b327-12d965dd47fe None admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:13.602494 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a45de61-01e7-46d4-b327-12d965dd47fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:13.602792 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a45de61-01e7-46d4-b327-12d965dd47fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:13.604197 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a45de61-01e7-46d4-b327-12d965dd47fe None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:13.621226 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 319/641] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:13.621226 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61da2e7-44e7-4b76-9915-8d1786580b12 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:13.621226 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61da2e7-44e7-4b76-9915-8d1786580b12 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:13.621226 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61da2e7-44e7-4b76-9915-8d1786580b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:13.634371 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-a61da2e7-44e7-4b76-9915-8d1786580b12 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:13.635579 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-a61da2e7-44e7-4b76-9915-8d1786580b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:13.698577 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54c8bc-831b-4764-b332-e5c3e947e702 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:13.701268 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 320/642] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:44:13.702090 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61da2e7-44e7-4b76-9915-8d1786580b12 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:13.749812 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b2499f-6d07-400b-a74d-3552e52be227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:13.750621 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b2499f-6d07-400b-a74d-3552e52be227 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:13.750911 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b2499f-6d07-400b-a74d-3552e52be227 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:13.751186 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b2499f-6d07-400b-a74d-3552e52be227 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:13.755701 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:13.759166 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b2499f-6d07-400b-a74d-3552e52be227 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:13.760714 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b2499f-6d07-400b-a74d-3552e52be227 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:13.775073 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-39b2499f-6d07-400b-a74d-3552e52be227 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 23 01:44:13.775073 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:13.775073 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:13.775073 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:13.775073 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:13.775073 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:13.775073 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:13.775073 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:13.775073 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:13.775073 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:13.775073 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:13.775073 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:13.775073 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 01:44:13.775073 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 01:44:13.775073 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 01:44:13.775073 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 01:44:13.776161 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:13.776161 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:13.776161 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 01:44:13.776161 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:13.776161 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 01:44:13.776161 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:13.776161 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:13.776161 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:13.776161 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:13.776161 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:13.776161 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:13.776161 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:13.776161 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:13.776161 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:13.776161 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:13.776161 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:13.776161 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:13.776161 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:13.777111 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:13.777111 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:13.777111 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 01:44:13.777111 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 01:44:13.777111 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 01:44:13.777111 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 01:44:13.777111 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 01:44:13.777111 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 01:44:13.777111 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 23 01:44:13.777111 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:13.777111 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 323/643] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:44:13 2022] GET /identity/v3/users/glance => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:13.777111 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b2499f-6d07-400b-a74d-3552e52be227 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:13.796187 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:13.817904 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61da2e7-44e7-4b76-9915-8d1786580b12 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:13.851678 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f6a138-6765-46a7-aa87-22094d3d06e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:13.852340 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6a138-6765-46a7-aa87-22094d3d06e6 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:13.852619 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6a138-6765-46a7-aa87-22094d3d06e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:13.852893 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6a138-6765-46a7-aa87-22094d3d06e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:13.853639 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f6a138-6765-46a7-aa87-22094d3d06e6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:13.853943 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f6a138-6765-46a7-aa87-22094d3d06e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:13.855389 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f6a138-6765-46a7-aa87-22094d3d06e6 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:13.863646 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 321/644] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Jun 23 01:44:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:13.864462 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f6a138-6765-46a7-aa87-22094d3d06e6 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:13.889244 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b2499f-6d07-400b-a74d-3552e52be227 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:13.925106 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a380dd-c80c-4f90-9852-7a9904779eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:13.925910 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a380dd-c80c-4f90-9852-7a9904779eec None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:13.926189 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a380dd-c80c-4f90-9852-7a9904779eec None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:13.926468 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a380dd-c80c-4f90-9852-7a9904779eec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:13.933742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a380dd-c80c-4f90-9852-7a9904779eec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:13.935300 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a380dd-c80c-4f90-9852-7a9904779eec None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:13.949622 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-91a380dd-c80c-4f90-9852-7a9904779eec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:13.949622 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:13.949622 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:13.949622 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:13.949622 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:13.949622 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:13.949622 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:13.949622 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:13.949622 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:13.949622 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:13.949622 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:13.949622 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:13.949622 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 01:44:13.949622 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 01:44:13.949622 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 01:44:13.950536 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 01:44:13.950536 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:13.950536 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:13.950536 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:13.950536 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:13.950536 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:13.950536 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:13.950536 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:13.950536 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:13.950536 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:13.950536 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:13.950536 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:13.950536 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:13.950536 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:13.950536 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:13.950536 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:13.950536 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:13.951615 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 01:44:13.951615 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 01:44:13.951615 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 01:44:13.951615 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 01:44:13.951615 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 01:44:13.951615 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 01:44:13.951615 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:13.951615 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:13.951615 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 324/645] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:44:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:13.951615 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a380dd-c80c-4f90-9852-7a9904779eec None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:13.971653 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f6a138-6765-46a7-aa87-22094d3d06e6 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:13.981829 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:14.009819 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8955f45b-42b7-4192-ad19-6c31d9e987e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:14.010811 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955f45b-42b7-4192-ad19-6c31d9e987e6 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:14.011100 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955f45b-42b7-4192-ad19-6c31d9e987e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:14.011736 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955f45b-42b7-4192-ad19-6c31d9e987e6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:14.024793 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8955f45b-42b7-4192-ad19-6c31d9e987e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:44:14.032664 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-8955f45b-42b7-4192-ad19-6c31d9e987e6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 01:44:14.032664 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:14.032664 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:14.032664 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:14.032664 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:14.032664 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:14.032664 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:14.032664 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:14.032664 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:14.032664 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:14.032664 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:14.032664 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:14.032664 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:44:14.032664 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:44:14.032664 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:44:14.032664 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:44:14.033595 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:44:14.033595 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:44:14.033595 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:44:14.033595 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:44:14.033595 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:14.033595 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:14.033595 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:14.033595 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:14.033595 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:14.033595 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:14.033595 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:14.033595 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:14.033595 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:14.033595 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:14.033595 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:14.033595 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:14.034416 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:14.034416 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:14.034416 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:14.034416 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:14.034416 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:44:14.034416 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:44:14.034416 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:44:14.034416 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:44:14.034416 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:44:14.034416 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:44:14.034416 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 01:44:14.034416 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:14.034416 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 322/646] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jun 23 01:44:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:14.035901 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8955f45b-42b7-4192-ad19-6c31d9e987e6 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:14.043663 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:14.070864 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a380dd-c80c-4f90-9852-7a9904779eec None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:14.106866 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450591c1-d877-4479-a16e-09d7edc6fcf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:14.107678 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450591c1-d877-4479-a16e-09d7edc6fcf4 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:14.107958 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450591c1-d877-4479-a16e-09d7edc6fcf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:14.108229 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450591c1-d877-4479-a16e-09d7edc6fcf4 None admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:14.109069 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450591c1-d877-4479-a16e-09d7edc6fcf4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:14.109378 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450591c1-d877-4479-a16e-09d7edc6fcf4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:14.111033 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450591c1-d877-4479-a16e-09d7edc6fcf4 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:14.129754 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 325/647] 10.222.0.26 () {56 vars in 1137 bytes} [Thu Jun 23 01:44:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:14.130626 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450591c1-d877-4479-a16e-09d7edc6fcf4 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:14.148232 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8955f45b-42b7-4192-ad19-6c31d9e987e6 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:14.184810 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd1c1d-78bc-4a59-bb88-8ec9a0da72ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:14.185873 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd1c1d-78bc-4a59-bb88-8ec9a0da72ee None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:14.186335 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd1c1d-78bc-4a59-bb88-8ec9a0da72ee None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:14.186709 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd1c1d-78bc-4a59-bb88-8ec9a0da72ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:14.202823 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd1c1d-78bc-4a59-bb88-8ec9a0da72ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:14.205189 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd1c1d-78bc-4a59-bb88-8ec9a0da72ee None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:14.227878 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-a8cd1c1d-78bc-4a59-bb88-8ec9a0da72ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:14.227878 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:14.227878 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:14.227878 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:14.227878 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:14.227878 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:14.227878 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:14.227878 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:14.227878 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:14.227878 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:14.227878 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:14.227878 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:14.227878 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:14.227878 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:14.227878 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:14.228806 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:14.228806 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:14.228806 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:14.228806 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:14.228806 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:14.228806 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:14.228806 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:14.228806 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:14.228806 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:14.228806 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:14.228806 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:14.228806 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:14.228806 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:14.228806 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:14.228806 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:14.228806 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:14.228806 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:14.229828 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:14.229828 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:14.229828 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:14.229828 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:14.231371 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 323/648] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:14.232267 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd1c1d-78bc-4a59-bb88-8ec9a0da72ee None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:14.239697 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450591c1-d877-4479-a16e-09d7edc6fcf4 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:14.255941 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:14.281538 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0068f83-ad36-402f-a557-c4039ae111d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:14.282212 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0068f83-ad36-402f-a557-c4039ae111d9 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:14.282494 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0068f83-ad36-402f-a557-c4039ae111d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:14.282775 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0068f83-ad36-402f-a557-c4039ae111d9 None admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:14.283656 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0068f83-ad36-402f-a557-c4039ae111d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:14.283967 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0068f83-ad36-402f-a557-c4039ae111d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:14.285436 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0068f83-ad36-402f-a557-c4039ae111d9 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:14.287963 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:14.299186 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 326/649] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jun 23 01:44:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:14.299953 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0068f83-ad36-402f-a557-c4039ae111d9 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:14.350487 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd1c1d-78bc-4a59-bb88-8ec9a0da72ee None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:14.384548 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b8e57-92dc-41fc-8724-8b1ccefa7c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:14.385214 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b8e57-92dc-41fc-8724-8b1ccefa7c0b None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:14.385499 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b8e57-92dc-41fc-8724-8b1ccefa7c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:14.385775 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b8e57-92dc-41fc-8724-8b1ccefa7c0b None admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:14.386531 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648b8e57-92dc-41fc-8724-8b1ccefa7c0b None admin] RBAC: Adding query filter params (name=service) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:14.386839 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648b8e57-92dc-41fc-8724-8b1ccefa7c0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:14.388265 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648b8e57-92dc-41fc-8724-8b1ccefa7c0b None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:14.400766 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 324/650] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Jun 23 01:44:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:14.400766 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b8e57-92dc-41fc-8724-8b1ccefa7c0b None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:14.407831 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0068f83-ad36-402f-a557-c4039ae111d9 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:14.443877 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe69198-019e-4c33-b701-926b4ee26aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:14.444528 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe69198-019e-4c33-b701-926b4ee26aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:14.444809 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe69198-019e-4c33-b701-926b4ee26aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:14.445081 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe69198-019e-4c33-b701-926b4ee26aa6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:14.482153 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:14.501458 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b8e57-92dc-41fc-8724-8b1ccefa7c0b None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:14.511571 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:14.520123 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe69198-019e-4c33-b701-926b4ee26aa6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:14.521592 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe69198-019e-4c33-b701-926b4ee26aa6 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:14.536211 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5de1450-60f9-4174-8a67-8b6b252ee366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:14.536875 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de1450-60f9-4174-8a67-8b6b252ee366 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:14.537156 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de1450-60f9-4174-8a67-8b6b252ee366 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:14.537432 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de1450-60f9-4174-8a67-8b6b252ee366 None admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:14.538168 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5de1450-60f9-4174-8a67-8b6b252ee366 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:14.538478 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5de1450-60f9-4174-8a67-8b6b252ee366 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:14.539898 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5de1450-60f9-4174-8a67-8b6b252ee366 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:14.551814 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 325/651] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:14.552610 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5de1450-60f9-4174-8a67-8b6b252ee366 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:14.632423 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 327/652] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:44:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 01:44:14.633578 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe69198-019e-4c33-b701-926b4ee26aa6 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:14.655199 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5de1450-60f9-4174-8a67-8b6b252ee366 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:14.698121 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c84574-13ff-405f-b397-9ece4afedaab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:14.698771 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c84574-13ff-405f-b397-9ece4afedaab None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:14.699052 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c84574-13ff-405f-b397-9ece4afedaab None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:14.699369 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c84574-13ff-405f-b397-9ece4afedaab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:14.700210 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c84574-13ff-405f-b397-9ece4afedaab None admin] RBAC: Adding query filter params (role.id=c20e054f00164967a4c16e7500c93361, scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c, user.id=ae4e83df33b64a3ebf28f54e2d94fe63) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:14.700654 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c84574-13ff-405f-b397-9ece4afedaab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:14.701960 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c84574-13ff-405f-b397-9ece4afedaab None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:14.715238 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 326/653] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:44:14 2022] GET /identity/v3/role_assignments?user.id=ae4e83df33b64a3ebf28f54e2d94fe63&scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c&role.id=c20e054f00164967a4c16e7500c93361 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:14.715238 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c84574-13ff-405f-b397-9ece4afedaab None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:14.719699 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:14.754247 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:14.762766 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe69198-019e-4c33-b701-926b4ee26aa6 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:14.811087 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a126f11-9035-408b-a30c-6d96e17990ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:14.812271 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a126f11-9035-408b-a30c-6d96e17990ec None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:14.813074 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a126f11-9035-408b-a30c-6d96e17990ec None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:14.813464 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a126f11-9035-408b-a30c-6d96e17990ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:14.828442 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a126f11-9035-408b-a30c-6d96e17990ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:14.830347 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a126f11-9035-408b-a30c-6d96e17990ec None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:14.834518 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c84574-13ff-405f-b397-9ece4afedaab None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:14.853340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-6a126f11-9035-408b-a30c-6d96e17990ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:14.853340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:14.853340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:14.853340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:14.853340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:14.853340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:14.853340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:14.853340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:14.853340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:14.853340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:14.853340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:14.853340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:14.853340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:14.853340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:14.853340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:14.854320 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:14.854320 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:14.854320 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:14.854320 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:14.854320 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:14.854320 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:14.854320 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:14.854320 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:14.854320 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:14.854320 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:14.854320 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:14.854320 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:14.854320 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:14.854320 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:14.854320 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:14.854320 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:14.854320 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:14.855319 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:14.855319 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:14.855319 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:14.855319 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:14.859577 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 328/654] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:14.860844 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a126f11-9035-408b-a30c-6d96e17990ec None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:14.889221 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b96415f-ab32-47dc-8415-f7270ca2dfef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:14.891963 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b96415f-ab32-47dc-8415-f7270ca2dfef None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:14.891963 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b96415f-ab32-47dc-8415-f7270ca2dfef None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:14.892246 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b96415f-ab32-47dc-8415-f7270ca2dfef None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:14.902668 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b96415f-ab32-47dc-8415-f7270ca2dfef None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:14.904552 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b96415f-ab32-47dc-8415-f7270ca2dfef None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:14.924554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-0b96415f-ab32-47dc-8415-f7270ca2dfef None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 01:44:14.924554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:14.924554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:14.924554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:14.924554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:14.924554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:14.924554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:14.924554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:14.924554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:14.924554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:14.924554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:14.924554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:14.924554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 01:44:14.924554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 01:44:14.924554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 01:44:14.924554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 01:44:14.925565 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:14.925565 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:14.925565 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 01:44:14.925565 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:14.925565 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 01:44:14.925565 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:14.925565 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:14.925565 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:14.925565 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:14.925565 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:14.925565 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:14.925565 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:14.925565 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:14.925565 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:14.925565 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:14.925565 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:14.925565 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:14.925565 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:14.926631 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:14.926631 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:14.926631 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 01:44:14.926631 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 01:44:14.926631 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 01:44:14.926631 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 01:44:14.926631 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 01:44:14.926631 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 01:44:14.926631 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 01:44:14.926631 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:14.928879 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 327/655] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jun 23 01:44:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:14.929902 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b96415f-ab32-47dc-8415-f7270ca2dfef None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:14.991078 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a126f11-9035-408b-a30c-6d96e17990ec None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:15.007690 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:15.028730 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b51e74b-6824-4ac5-ac19-b8dd256d37a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:15.029536 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51e74b-6824-4ac5-ac19-b8dd256d37a9 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:15.029817 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51e74b-6824-4ac5-ac19-b8dd256d37a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:15.030087 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51e74b-6824-4ac5-ac19-b8dd256d37a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:15.040406 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b51e74b-6824-4ac5-ac19-b8dd256d37a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:15.041869 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b51e74b-6824-4ac5-ac19-b8dd256d37a9 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:15.042617 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b96415f-ab32-47dc-8415-f7270ca2dfef None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:15.050784 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:15.061684 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-1b51e74b-6824-4ac5-ac19-b8dd256d37a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:15.061684 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:15.061684 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:15.061684 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:15.061684 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:15.061684 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:15.061684 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:15.061684 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:15.061684 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:15.061684 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:15.061684 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:15.061684 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:15.061684 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:15.061684 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:15.061684 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:15.062783 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:15.062783 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:15.062783 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:15.062783 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:15.062783 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:15.062783 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:15.062783 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:15.062783 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:15.062783 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:15.062783 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:15.062783 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:15.062783 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:15.062783 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:15.062783 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:15.062783 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:15.062783 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:15.062783 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:15.063735 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:15.063735 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:15.063735 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:15.063735 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:15.070513 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 329/656] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:15.070513 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b51e74b-6824-4ac5-ac19-b8dd256d37a9 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:15.083175 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527aa154-8d27-4b21-a7ee-478991206ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:15.083964 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527aa154-8d27-4b21-a7ee-478991206ca5 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:15.084307 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527aa154-8d27-4b21-a7ee-478991206ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:15.084642 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527aa154-8d27-4b21-a7ee-478991206ca5 None admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:15.085436 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527aa154-8d27-4b21-a7ee-478991206ca5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:15.085797 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527aa154-8d27-4b21-a7ee-478991206ca5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:15.087174 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527aa154-8d27-4b21-a7ee-478991206ca5 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:15.098526 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 328/657] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:15.099876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7843e15f-d4f1-4f89-a608-db3ab986fae0 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:15.100160 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7843e15f-d4f1-4f89-a608-db3ab986fae0 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:15.100431 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7843e15f-d4f1-4f89-a608-db3ab986fae0 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:15.105772 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 329/658] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:15.106553 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7843e15f-d4f1-4f89-a608-db3ab986fae0 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:15.172776 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b51e74b-6824-4ac5-ac19-b8dd256d37a9 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:15.205996 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7843e15f-d4f1-4f89-a608-db3ab986fae0 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:15.207263 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8207b1c2-69ee-42bf-a74a-120ac203d07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:15.207943 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207b1c2-69ee-42bf-a74a-120ac203d07d None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:15.208224 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207b1c2-69ee-42bf-a74a-120ac203d07d None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:15.208500 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207b1c2-69ee-42bf-a74a-120ac203d07d None admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:15.209347 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8207b1c2-69ee-42bf-a74a-120ac203d07d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:15.209648 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8207b1c2-69ee-42bf-a74a-120ac203d07d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:15.211101 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8207b1c2-69ee-42bf-a74a-120ac203d07d None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:15.230117 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 330/659] 10.222.0.26 () {56 vars in 1139 bytes} [Thu Jun 23 01:44:15 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:15.231539 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:15.235373 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8207b1c2-69ee-42bf-a74a-120ac203d07d None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:15.241912 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8556e2-a6ad-4ada-a9b1-69910e4e75e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:15.242574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8556e2-a6ad-4ada-a9b1-69910e4e75e8 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:15.242855 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8556e2-a6ad-4ada-a9b1-69910e4e75e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:15.243123 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8556e2-a6ad-4ada-a9b1-69910e4e75e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:15.243897 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8556e2-a6ad-4ada-a9b1-69910e4e75e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:15.244196 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8556e2-a6ad-4ada-a9b1-69910e4e75e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:15.245576 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8556e2-a6ad-4ada-a9b1-69910e4e75e8 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:15.261915 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 330/660] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:15.261915 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63730efd-c285-46b2-bf62-e18803045522 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:15.261915 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63730efd-c285-46b2-bf62-e18803045522 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:15.261915 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63730efd-c285-46b2-bf62-e18803045522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:15.277881 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-63730efd-c285-46b2-bf62-e18803045522 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:15.279071 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-63730efd-c285-46b2-bf62-e18803045522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:15.308264 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:15.346579 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 331/661] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:44:15.347297 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63730efd-c285-46b2-bf62-e18803045522 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:15.360184 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8207b1c2-69ee-42bf-a74a-120ac203d07d None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:15.404228 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba9a83-e568-4814-bd82-b8503123aad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:15.405034 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba9a83-e568-4814-bd82-b8503123aad8 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:15.405310 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba9a83-e568-4814-bd82-b8503123aad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:15.405585 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba9a83-e568-4814-bd82-b8503123aad8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:15.413680 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ba9a83-e568-4814-bd82-b8503123aad8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:15.415142 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ba9a83-e568-4814-bd82-b8503123aad8 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:15.430750 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-73ba9a83-e568-4814-bd82-b8503123aad8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 01:44:15.430750 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:15.430750 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:15.430750 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:15.430750 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:15.430750 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:15.430750 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:15.430750 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:15.430750 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:15.430750 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:15.430750 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:15.430750 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:15.430750 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 01:44:15.430750 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 01:44:15.430750 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 01:44:15.430750 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 01:44:15.431889 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:15.431889 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:15.431889 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 01:44:15.431889 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:15.431889 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 01:44:15.431889 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:15.431889 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:15.431889 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:15.431889 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:15.431889 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:15.431889 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:15.431889 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:15.431889 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:15.431889 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:15.431889 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:15.431889 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:15.431889 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:15.431889 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:15.432820 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:15.432820 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:15.432820 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 01:44:15.432820 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 01:44:15.432820 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 01:44:15.432820 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 01:44:15.432820 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 01:44:15.432820 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 01:44:15.432820 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 01:44:15.432820 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:15.432820 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 331/662] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:44:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:15.432820 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba9a83-e568-4814-bd82-b8503123aad8 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:15.452300 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63730efd-c285-46b2-bf62-e18803045522 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:15.468511 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:15.488877 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3b0522-227b-4dc5-8596-7f88f2b23b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:15.489541 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b0522-227b-4dc5-8596-7f88f2b23b0b None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:15.489828 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b0522-227b-4dc5-8596-7f88f2b23b0b None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:15.490108 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b0522-227b-4dc5-8596-7f88f2b23b0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:15.547029 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba9a83-e568-4814-bd82-b8503123aad8 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:15.559379 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3b0522-227b-4dc5-8596-7f88f2b23b0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:15.561003 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3b0522-227b-4dc5-8596-7f88f2b23b0b None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:15.565917 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:15.584043 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54be824-ee79-44b5-b0f7-44b024789da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:15.584841 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54be824-ee79-44b5-b0f7-44b024789da1 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:15.585118 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54be824-ee79-44b5-b0f7-44b024789da1 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:15.585386 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54be824-ee79-44b5-b0f7-44b024789da1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:15.592524 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54be824-ee79-44b5-b0f7-44b024789da1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:15.593963 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54be824-ee79-44b5-b0f7-44b024789da1 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:15.607752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-a54be824-ee79-44b5-b0f7-44b024789da1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:15.607752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:15.607752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:15.607752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:15.607752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:15.607752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:15.607752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:15.607752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:15.607752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:15.607752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:15.607752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:15.607752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:15.607752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 01:44:15.607752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 01:44:15.607752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 01:44:15.608726 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 01:44:15.608726 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:15.608726 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:15.608726 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:15.608726 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:15.608726 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:15.608726 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:15.608726 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:15.608726 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:15.608726 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:15.608726 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:15.608726 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:15.608726 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:15.608726 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:15.608726 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:15.608726 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:15.608726 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:15.610238 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 01:44:15.610238 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 01:44:15.610238 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 01:44:15.610238 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 01:44:15.610238 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 01:44:15.610238 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 01:44:15.610238 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:15.610238 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:15.610238 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 332/663] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:44:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:15.610238 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54be824-ee79-44b5-b0f7-44b024789da1 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:15.668173 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 332/664] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:44:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 01:44:15.669226 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3b0522-227b-4dc5-8596-7f88f2b23b0b None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:15.686327 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:15.720574 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54be824-ee79-44b5-b0f7-44b024789da1 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:15.761730 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e903adc-eda5-49ff-be1b-8759c399e9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:15.762532 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e903adc-eda5-49ff-be1b-8759c399e9fd None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:15.762804 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e903adc-eda5-49ff-be1b-8759c399e9fd None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:15.763074 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e903adc-eda5-49ff-be1b-8759c399e9fd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:15.774061 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e903adc-eda5-49ff-be1b-8759c399e9fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:44:15.780464 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3b0522-227b-4dc5-8596-7f88f2b23b0b None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:15.781907 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-8e903adc-eda5-49ff-be1b-8759c399e9fd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 01:44:15.781907 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:15.781907 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:15.781907 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:15.781907 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:15.781907 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:15.781907 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:15.781907 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:15.781907 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:15.781907 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:15.781907 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:15.781907 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:15.781907 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:44:15.781907 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:44:15.781907 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:44:15.781907 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:44:15.782819 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:44:15.782819 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:44:15.782819 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:44:15.782819 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:44:15.782819 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:15.782819 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:15.782819 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:15.782819 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:15.782819 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:15.782819 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:15.782819 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:15.782819 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:15.782819 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:15.782819 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:15.782819 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:15.782819 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:15.783681 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:15.783681 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:15.783681 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:15.783681 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:15.783681 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:44:15.783681 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:44:15.783681 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:44:15.783681 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:44:15.783681 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:44:15.783681 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:44:15.783681 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 01:44:15.783681 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:15.783681 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 333/665] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jun 23 01:44:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:15.784354 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e903adc-eda5-49ff-be1b-8759c399e9fd None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:15.818555 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addbdfe3-ec80-421d-a3d9-74664a179b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:15.819143 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addbdfe3-ec80-421d-a3d9-74664a179b2e None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:15.819462 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addbdfe3-ec80-421d-a3d9-74664a179b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:15.819823 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addbdfe3-ec80-421d-a3d9-74664a179b2e None admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:15.820559 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addbdfe3-ec80-421d-a3d9-74664a179b2e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:15.820860 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addbdfe3-ec80-421d-a3d9-74664a179b2e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:15.822308 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addbdfe3-ec80-421d-a3d9-74664a179b2e None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:15.831573 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:15.842516 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 333/666] 10.222.0.26 () {56 vars in 1137 bytes} [Thu Jun 23 01:44:15 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:15.843487 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addbdfe3-ec80-421d-a3d9-74664a179b2e None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:15.904676 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e903adc-eda5-49ff-be1b-8759c399e9fd None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:15.916948 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:15.942958 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdeb0a9-c174-476b-aff3-080340728e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:15.943661 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdeb0a9-c174-476b-aff3-080340728e1e None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:15.943966 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdeb0a9-c174-476b-aff3-080340728e1e None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:15.944241 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdeb0a9-c174-476b-aff3-080340728e1e None admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:15.944434 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addbdfe3-ec80-421d-a3d9-74664a179b2e None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:15.945037 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdeb0a9-c174-476b-aff3-080340728e1e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:15.945356 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdeb0a9-c174-476b-aff3-080340728e1e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:15.946874 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdeb0a9-c174-476b-aff3-080340728e1e None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:15.964572 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 334/667] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jun 23 01:44:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:15.964572 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdeb0a9-c174-476b-aff3-080340728e1e None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:15.984809 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5417470b-783d-4d64-a0da-eda0f5d0b8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:15.985595 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5417470b-783d-4d64-a0da-eda0f5d0b8f4 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:15.985872 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5417470b-783d-4d64-a0da-eda0f5d0b8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:15.986143 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5417470b-783d-4d64-a0da-eda0f5d0b8f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:15.996324 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5417470b-783d-4d64-a0da-eda0f5d0b8f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:15.997772 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5417470b-783d-4d64-a0da-eda0f5d0b8f4 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:16.014817 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-5417470b-783d-4d64-a0da-eda0f5d0b8f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:16.014817 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:16.014817 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:16.014817 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:16.014817 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:16.014817 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:16.014817 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:16.014817 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:16.014817 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:16.014817 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:16.014817 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:16.014817 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:16.014817 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:16.014817 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:16.014817 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:16.015781 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:16.015781 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:16.015781 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:16.015781 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:16.015781 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:16.015781 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:16.015781 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:16.015781 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:16.015781 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:16.015781 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:16.015781 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:16.015781 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:16.015781 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:16.015781 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:16.015781 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:16.015781 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:16.015781 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:16.016661 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:16.016661 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:16.016661 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:16.016661 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:16.016661 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 334/668] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:16.016952 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5417470b-783d-4d64-a0da-eda0f5d0b8f4 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:16.060753 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:16.064809 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdeb0a9-c174-476b-aff3-080340728e1e None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:16.101352 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2679d31-e16d-4faf-89f3-a480115bb51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:16.101962 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2679d31-e16d-4faf-89f3-a480115bb51a None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:16.102246 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2679d31-e16d-4faf-89f3-a480115bb51a None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:16.102520 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2679d31-e16d-4faf-89f3-a480115bb51a None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:16.103298 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2679d31-e16d-4faf-89f3-a480115bb51a None admin] RBAC: Adding query filter params (name=service) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:16.103608 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2679d31-e16d-4faf-89f3-a480115bb51a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:16.104987 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2679d31-e16d-4faf-89f3-a480115bb51a None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:16.113631 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 335/669] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Jun 23 01:44:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:16.114414 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2679d31-e16d-4faf-89f3-a480115bb51a None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:16.128922 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5417470b-783d-4d64-a0da-eda0f5d0b8f4 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:16.156564 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:16.167593 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e0e7d-33e2-41fb-940e-7a3f0b005d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:16.168245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e0e7d-33e2-41fb-940e-7a3f0b005d26 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:16.168530 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e0e7d-33e2-41fb-940e-7a3f0b005d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:16.168804 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447e0e7d-33e2-41fb-940e-7a3f0b005d26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:16.222939 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2679d31-e16d-4faf-89f3-a480115bb51a None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:16.240784 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447e0e7d-33e2-41fb-940e-7a3f0b005d26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:16.242369 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447e0e7d-33e2-41fb-940e-7a3f0b005d26 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:16.256742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a57b5f0-06be-431c-8580-342217f89fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:16.257401 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a57b5f0-06be-431c-8580-342217f89fd5 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:16.257680 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a57b5f0-06be-431c-8580-342217f89fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:16.257954 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a57b5f0-06be-431c-8580-342217f89fd5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:16.258783 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a57b5f0-06be-431c-8580-342217f89fd5 None admin] RBAC: Adding query filter params (role.id=c20e054f00164967a4c16e7500c93361, scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c, user.id=1c081b4cf34146ca8eb197b62a133433) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:16.259092 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a57b5f0-06be-431c-8580-342217f89fd5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:16.260559 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a57b5f0-06be-431c-8580-342217f89fd5 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:16.268306 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 336/670] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:44:16 2022] GET /identity/v3/role_assignments?user.id=1c081b4cf34146ca8eb197b62a133433&scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c&role.id=c20e054f00164967a4c16e7500c93361 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:16.269216 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a57b5f0-06be-431c-8580-342217f89fd5 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:16.303091 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:16.397822 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 335/671] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:44:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 01:44:16.398379 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a57b5f0-06be-431c-8580-342217f89fd5 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:16.399197 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e0e7d-33e2-41fb-940e-7a3f0b005d26 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:16.437460 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:16.438320 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e36cf4-2c09-471f-aa7f-5d338a048513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:16.439998 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e36cf4-2c09-471f-aa7f-5d338a048513 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:16.440517 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e36cf4-2c09-471f-aa7f-5d338a048513 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:16.441354 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e36cf4-2c09-471f-aa7f-5d338a048513 None admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:16.442557 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e36cf4-2c09-471f-aa7f-5d338a048513 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:16.443151 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e36cf4-2c09-471f-aa7f-5d338a048513 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:16.444831 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e36cf4-2c09-471f-aa7f-5d338a048513 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:16.462828 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 337/672] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:16.463993 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e36cf4-2c09-471f-aa7f-5d338a048513 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:16.525474 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447e0e7d-33e2-41fb-940e-7a3f0b005d26 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:16.563460 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c0ff1f-7b3d-4ba4-9359-456a3edabdaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:16.564314 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:16.564455 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c0ff1f-7b3d-4ba4-9359-456a3edabdaa None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:16.564854 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c0ff1f-7b3d-4ba4-9359-456a3edabdaa None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:16.565251 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c0ff1f-7b3d-4ba4-9359-456a3edabdaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:16.577540 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c0ff1f-7b3d-4ba4-9359-456a3edabdaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:16.579149 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c0ff1f-7b3d-4ba4-9359-456a3edabdaa None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:16.590786 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e36cf4-2c09-471f-aa7f-5d338a048513 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:16.599387 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-97c0ff1f-7b3d-4ba4-9359-456a3edabdaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:16.599387 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:16.599387 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:16.599387 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:16.599387 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:16.599387 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:16.599387 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:16.599387 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:16.599387 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:16.599387 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:16.599387 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:16.599387 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:16.599387 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:16.599387 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:16.599387 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:16.600559 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:16.600559 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:16.600559 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:16.600559 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:16.600559 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:16.600559 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:16.600559 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:16.600559 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:16.600559 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:16.600559 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:16.600559 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:16.600559 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:16.600559 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:16.600559 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:16.600559 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:16.600559 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:16.600559 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:16.601481 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:16.601481 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:16.601481 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:16.601481 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:16.604954 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 336/673] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:16.605897 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c0ff1f-7b3d-4ba4-9359-456a3edabdaa None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:16.636523 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e61d84-4956-439c-8d60-8db2fb75efdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:16.637718 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e61d84-4956-439c-8d60-8db2fb75efdd None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:16.638402 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e61d84-4956-439c-8d60-8db2fb75efdd None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:16.639230 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e61d84-4956-439c-8d60-8db2fb75efdd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:16.658505 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e61d84-4956-439c-8d60-8db2fb75efdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:16.660678 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e61d84-4956-439c-8d60-8db2fb75efdd None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:16.685651 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-78e61d84-4956-439c-8d60-8db2fb75efdd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:16.685651 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:16.685651 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:16.685651 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:16.685651 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:16.685651 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:16.685651 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:16.685651 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:16.685651 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:16.685651 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:16.685651 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:16.685651 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:16.685651 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:16.685651 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:16.685651 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:16.686796 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:16.686796 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:16.686796 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:16.686796 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:16.686796 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:16.686796 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:16.686796 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:16.686796 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:16.686796 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:16.686796 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:16.686796 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:16.686796 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:16.686796 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:16.686796 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:16.686796 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:16.686796 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:16.686796 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:16.687925 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:16.687925 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:16.687925 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:16.687925 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:16.690863 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:16.691066 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 338/674] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:16.693153 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e61d84-4956-439c-8d60-8db2fb75efdd None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:16.724346 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c0ff1f-7b3d-4ba4-9359-456a3edabdaa None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:16.771427 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab0551-0103-4851-bd48-3a5b8248e537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:16.772894 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab0551-0103-4851-bd48-3a5b8248e537 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:16.773327 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab0551-0103-4851-bd48-3a5b8248e537 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:16.773745 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab0551-0103-4851-bd48-3a5b8248e537 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:16.783364 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffab0551-0103-4851-bd48-3a5b8248e537 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:16.785071 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffab0551-0103-4851-bd48-3a5b8248e537 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:16.800167 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-ffab0551-0103-4851-bd48-3a5b8248e537 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:16.800167 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:16.800167 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:16.800167 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:16.800167 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:16.800167 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:16.800167 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:16.800167 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:16.800167 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:16.800167 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:16.800167 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:16.800167 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:16.800167 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 01:44:16.800167 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 01:44:16.800167 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 01:44:16.801290 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 01:44:16.801290 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:16.801290 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:16.801290 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:16.801290 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:16.801290 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:16.801290 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:16.801290 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:16.801290 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:16.801290 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:16.801290 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:16.801290 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:16.801290 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:16.801290 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:16.801290 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:16.801290 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:16.801290 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:16.802398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 01:44:16.802398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 01:44:16.802398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 01:44:16.802398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 01:44:16.802398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 01:44:16.802398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 01:44:16.802398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:16.802398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:16.803679 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 337/675] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:44:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:16.804679 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab0551-0103-4851-bd48-3a5b8248e537 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:16.805754 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e61d84-4956-439c-8d60-8db2fb75efdd None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:16.848894 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b25013-88ae-41c0-8a0c-62c0016de5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:16.849852 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b25013-88ae-41c0-8a0c-62c0016de5f3 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:16.850133 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b25013-88ae-41c0-8a0c-62c0016de5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:16.850406 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b25013-88ae-41c0-8a0c-62c0016de5f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:16.851139 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b25013-88ae-41c0-8a0c-62c0016de5f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:16.851476 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b25013-88ae-41c0-8a0c-62c0016de5f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:16.852865 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b25013-88ae-41c0-8a0c-62c0016de5f3 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:16.855268 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:16.866416 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 339/676] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:16.867497 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b25013-88ae-41c0-8a0c-62c0016de5f3 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:16.921358 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab0551-0103-4851-bd48-3a5b8248e537 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:16.955964 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d18ff5f-56c2-47cf-89cd-3739a61476aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:16.956908 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18ff5f-56c2-47cf-89cd-3739a61476aa None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:16.957186 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18ff5f-56c2-47cf-89cd-3739a61476aa None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:16.957457 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18ff5f-56c2-47cf-89cd-3739a61476aa None admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:16.958192 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d18ff5f-56c2-47cf-89cd-3739a61476aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:16.958495 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d18ff5f-56c2-47cf-89cd-3739a61476aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:16.959904 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d18ff5f-56c2-47cf-89cd-3739a61476aa None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:16.966755 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:16.972469 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 338/677] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:16.973846 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d60c84-af02-493b-a0ed-44979a8d5495 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:16.974223 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d60c84-af02-493b-a0ed-44979a8d5495 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:16.974589 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d60c84-af02-493b-a0ed-44979a8d5495 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:16.976067 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 339/678] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:16.980566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d60c84-af02-493b-a0ed-44979a8d5495 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:16.980900 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b25013-88ae-41c0-8a0c-62c0016de5f3 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:17.017859 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cd6354-be87-4143-a1c2-85351028125d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:17.018772 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd6354-be87-4143-a1c2-85351028125d None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:17.019315 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd6354-be87-4143-a1c2-85351028125d None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:17.019819 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd6354-be87-4143-a1c2-85351028125d None admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:17.020860 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cd6354-be87-4143-a1c2-85351028125d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:17.021452 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cd6354-be87-4143-a1c2-85351028125d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:17.023021 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cd6354-be87-4143-a1c2-85351028125d None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:17.040831 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 340/679] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jun 23 01:44:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:17.040831 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cd6354-be87-4143-a1c2-85351028125d None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:17.086825 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d60c84-af02-493b-a0ed-44979a8d5495 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:17.103711 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:17.125699 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddb5c5d-0ceb-4f2f-a839-81b914a980a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:17.126495 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb5c5d-0ceb-4f2f-a839-81b914a980a4 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:17.127162 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb5c5d-0ceb-4f2f-a839-81b914a980a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:17.127463 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb5c5d-0ceb-4f2f-a839-81b914a980a4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:17.135271 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddb5c5d-0ceb-4f2f-a839-81b914a980a4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:17.136722 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddb5c5d-0ceb-4f2f-a839-81b914a980a4 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:17.149145 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cd6354-be87-4143-a1c2-85351028125d None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:17.151592 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-5ddb5c5d-0ceb-4f2f-a839-81b914a980a4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 01:44:17.151592 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:17.151592 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:17.151592 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:17.151592 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:17.151592 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:17.151592 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:17.151592 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:17.151592 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:17.151592 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:17.151592 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:17.151592 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:17.151592 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 01:44:17.151592 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 01:44:17.151592 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 01:44:17.151592 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 01:44:17.152508 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:17.152508 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:17.152508 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 01:44:17.152508 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:17.152508 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 01:44:17.152508 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:17.152508 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:17.152508 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:17.152508 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:17.152508 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:17.152508 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:17.152508 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:17.152508 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:17.152508 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:17.152508 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:17.152508 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:17.152508 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:17.152508 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:17.153408 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:17.153408 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:17.153408 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 01:44:17.153408 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 01:44:17.153408 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 01:44:17.153408 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 01:44:17.153408 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 01:44:17.153408 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 01:44:17.153408 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 01:44:17.153408 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:17.153408 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 340/680] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jun 23 01:44:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:17.154010 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b0a8d9-dd74-456b-b75a-02a2e1ac92c1 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:17.154408 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b0a8d9-dd74-456b-b75a-02a2e1ac92c1 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:17.154562 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b0a8d9-dd74-456b-b75a-02a2e1ac92c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:17.172764 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-36b0a8d9-dd74-456b-b75a-02a2e1ac92c1 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:17.173898 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-36b0a8d9-dd74-456b-b75a-02a2e1ac92c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:17.188771 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21942b19-766e-434e-9451-801ff88e808f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:17.189567 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21942b19-766e-434e-9451-801ff88e808f None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:17.189846 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21942b19-766e-434e-9451-801ff88e808f None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:17.190172 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21942b19-766e-434e-9451-801ff88e808f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:17.197900 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21942b19-766e-434e-9451-801ff88e808f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:17.199495 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21942b19-766e-434e-9451-801ff88e808f None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:17.214986 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-21942b19-766e-434e-9451-801ff88e808f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:17.214986 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:17.214986 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:17.214986 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:17.214986 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:17.214986 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:17.214986 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:17.214986 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:17.214986 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:17.214986 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:17.214986 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:17.214986 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:17.214986 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 01:44:17.214986 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 01:44:17.214986 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 01:44:17.216032 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 01:44:17.216032 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:17.216032 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:17.216032 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:17.216032 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:17.216032 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:17.216032 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:17.216032 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:17.216032 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:17.216032 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:17.216032 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:17.216032 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:17.216032 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:17.216032 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:17.216032 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:17.216032 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:17.216032 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:17.216979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 01:44:17.216979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 01:44:17.216979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 01:44:17.216979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 01:44:17.216979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 01:44:17.216979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 01:44:17.216979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:17.216979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:17.216979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 341/681] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:44:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:17.216979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21942b19-766e-434e-9451-801ff88e808f None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:17.234146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:17.249221 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 341/682] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:44:17.250028 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b0a8d9-dd74-456b-b75a-02a2e1ac92c1 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:17.334719 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21942b19-766e-434e-9451-801ff88e808f None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:17.352951 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:17.366956 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b0a8d9-dd74-456b-b75a-02a2e1ac92c1 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:17.372560 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41251939-4f4f-40ae-9c8b-bb360c440fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:17.373226 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41251939-4f4f-40ae-9c8b-bb360c440fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:17.373508 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41251939-4f4f-40ae-9c8b-bb360c440fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:17.373783 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41251939-4f4f-40ae-9c8b-bb360c440fe5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:17.374614 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41251939-4f4f-40ae-9c8b-bb360c440fe5 None admin] RBAC: Adding query filter params (role.id=c20e054f00164967a4c16e7500c93361, scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c, user.id=948e4905de44473bb7a45e1b521cf095) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:17.374932 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41251939-4f4f-40ae-9c8b-bb360c440fe5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:17.376424 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41251939-4f4f-40ae-9c8b-bb360c440fe5 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:17.382639 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 342/683] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:44:17 2022] GET /identity/v3/role_assignments?user.id=948e4905de44473bb7a45e1b521cf095&scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c&role.id=c20e054f00164967a4c16e7500c93361 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:17.383457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41251939-4f4f-40ae-9c8b-bb360c440fe5 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:17.416913 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ddcc7c-9ddf-46a1-9d02-07d8969bc35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:17.417600 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ddcc7c-9ddf-46a1-9d02-07d8969bc35f None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:17.417879 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ddcc7c-9ddf-46a1-9d02-07d8969bc35f None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:17.418153 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ddcc7c-9ddf-46a1-9d02-07d8969bc35f None admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:17.419088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ddcc7c-9ddf-46a1-9d02-07d8969bc35f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:17.419434 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ddcc7c-9ddf-46a1-9d02-07d8969bc35f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:17.420996 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ddcc7c-9ddf-46a1-9d02-07d8969bc35f None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:17.445771 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 342/684] 10.222.0.26 () {56 vars in 1135 bytes} [Thu Jun 23 01:44:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:17.445771 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ddcc7c-9ddf-46a1-9d02-07d8969bc35f None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:17.500070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:17.521544 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41251939-4f4f-40ae-9c8b-bb360c440fe5 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:17.561262 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd28a6d7-6961-4066-bdee-1cb81ffc92d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:17.561940 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28a6d7-6961-4066-bdee-1cb81ffc92d0 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:17.562223 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28a6d7-6961-4066-bdee-1cb81ffc92d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:17.562498 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28a6d7-6961-4066-bdee-1cb81ffc92d0 None admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:17.563357 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd28a6d7-6961-4066-bdee-1cb81ffc92d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:17.563674 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd28a6d7-6961-4066-bdee-1cb81ffc92d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:17.565145 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd28a6d7-6961-4066-bdee-1cb81ffc92d0 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:17.567937 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ddcc7c-9ddf-46a1-9d02-07d8969bc35f None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:17.584517 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 343/685] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jun 23 01:44:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:17.584517 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd28a6d7-6961-4066-bdee-1cb81ffc92d0 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:17.615004 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e076c3a6-a2fb-40a0-89fe-1677cb67b02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:17.615828 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076c3a6-a2fb-40a0-89fe-1677cb67b02b None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:17.616114 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076c3a6-a2fb-40a0-89fe-1677cb67b02b None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:17.616383 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e076c3a6-a2fb-40a0-89fe-1677cb67b02b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:17.626410 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e076c3a6-a2fb-40a0-89fe-1677cb67b02b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:44:17.634225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-e076c3a6-a2fb-40a0-89fe-1677cb67b02b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:44:17.634225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:17.634225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:17.634225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:17.634225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:17.634225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:17.634225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:17.634225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:17.634225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:17.634225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:17.634225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:17.634225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:17.634225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:44:17.634225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:44:17.634225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:44:17.634225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:44:17.635176 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:44:17.635176 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:44:17.635176 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:44:17.635176 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:44:17.635176 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:17.635176 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:17.635176 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:17.635176 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:17.635176 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:17.635176 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:17.635176 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:17.635176 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:17.635176 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:17.635176 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:17.635176 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:17.635176 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:17.636058 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:17.636058 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:17.636058 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:17.636058 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:17.636058 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:44:17.636058 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:44:17.636058 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:44:17.636058 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:44:17.636058 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:44:17.636058 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:44:17.636058 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:44:17.636058 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:17.636685 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 343/686] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jun 23 01:44:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:17.637322 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e076c3a6-a2fb-40a0-89fe-1677cb67b02b None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:17.643350 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:17.712513 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd28a6d7-6961-4066-bdee-1cb81ffc92d0 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:17.755869 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6558d56a-23b1-4631-922b-4d67920a3614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:17.757283 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6558d56a-23b1-4631-922b-4d67920a3614 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:17.758054 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6558d56a-23b1-4631-922b-4d67920a3614 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:17.758533 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6558d56a-23b1-4631-922b-4d67920a3614 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:17.759845 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e076c3a6-a2fb-40a0-89fe-1677cb67b02b None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:17.767046 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:17.796103 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16045c6-31f5-46e7-957e-406e3a77a84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:17.797022 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16045c6-31f5-46e7-957e-406e3a77a84a None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:17.797445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16045c6-31f5-46e7-957e-406e3a77a84a None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:17.797870 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16045c6-31f5-46e7-957e-406e3a77a84a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:17.799005 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16045c6-31f5-46e7-957e-406e3a77a84a None admin] RBAC: Adding query filter params (role.id=c20e054f00164967a4c16e7500c93361, scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c, user.id=dc5bb02f8eb64ac980e3218470b3c1b0) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:17.799590 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16045c6-31f5-46e7-957e-406e3a77a84a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:17.801355 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16045c6-31f5-46e7-957e-406e3a77a84a None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:17.815299 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 344/687] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:44:17 2022] GET /identity/v3/role_assignments?user.id=dc5bb02f8eb64ac980e3218470b3c1b0&scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c&role.id=c20e054f00164967a4c16e7500c93361 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:17.816770 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16045c6-31f5-46e7-957e-406e3a77a84a None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:17.845462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6558d56a-23b1-4631-922b-4d67920a3614 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:17.847563 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6558d56a-23b1-4631-922b-4d67920a3614 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:17.899461 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:17.979758 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 344/688] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:44:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 01:44:17.982261 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6558d56a-23b1-4631-922b-4d67920a3614 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:18.056343 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16045c6-31f5-46e7-957e-406e3a77a84a None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:18.102022 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aced75-edfd-451c-91e3-9ab76612a01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:18.103206 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aced75-edfd-451c-91e3-9ab76612a01e None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:18.103683 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aced75-edfd-451c-91e3-9ab76612a01e None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:18.104106 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aced75-edfd-451c-91e3-9ab76612a01e None admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:18.105016 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aced75-edfd-451c-91e3-9ab76612a01e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:18.105130 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6558d56a-23b1-4631-922b-4d67920a3614 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:18.105563 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aced75-edfd-451c-91e3-9ab76612a01e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:18.107326 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aced75-edfd-451c-91e3-9ab76612a01e None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:18.118889 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 345/689] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:44:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:18.125714 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aced75-edfd-451c-91e3-9ab76612a01e None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:18.147162 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a2054a-5427-46ef-a740-bbbe3b8058e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:18.148512 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2054a-5427-46ef-a740-bbbe3b8058e5 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:18.148664 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2054a-5427-46ef-a740-bbbe3b8058e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:18.149034 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2054a-5427-46ef-a740-bbbe3b8058e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:18.158307 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:18.161300 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a2054a-5427-46ef-a740-bbbe3b8058e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:18.162874 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a2054a-5427-46ef-a740-bbbe3b8058e5 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:18.167951 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:18.189331 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-08a2054a-5427-46ef-a740-bbbe3b8058e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:18.189331 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:18.189331 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:18.189331 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:18.189331 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:18.189331 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:18.189331 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:18.189331 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:18.189331 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:18.189331 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:18.189331 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:18.189331 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:18.189331 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:18.189331 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:18.189331 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:18.190290 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:18.190290 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:18.190290 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:18.190290 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:18.190290 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:18.190290 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:18.190290 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:18.190290 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:18.190290 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:18.190290 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:18.190290 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:18.190290 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:18.190290 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:18.190290 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:18.190290 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:18.190290 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:18.190290 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:18.191245 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:18.191245 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:18.191245 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:18.191245 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:18.191245 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 345/690] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:44:18.197344 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a2054a-5427-46ef-a740-bbbe3b8058e5 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:18.256405 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aced75-edfd-451c-91e3-9ab76612a01e None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:18.297835 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164145a4-7fce-4a68-9967-4b83be621067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:18.298999 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164145a4-7fce-4a68-9967-4b83be621067 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:18.299483 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164145a4-7fce-4a68-9967-4b83be621067 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:18.300183 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164145a4-7fce-4a68-9967-4b83be621067 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:18.314429 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164145a4-7fce-4a68-9967-4b83be621067 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:18.316554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164145a4-7fce-4a68-9967-4b83be621067 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:18.333329 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a2054a-5427-46ef-a740-bbbe3b8058e5 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:18.337381 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-164145a4-7fce-4a68-9967-4b83be621067 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:18.337381 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:18.337381 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:18.337381 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:18.337381 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:18.337381 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:18.337381 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:18.337381 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:18.337381 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:18.337381 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:18.337381 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:18.337381 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:18.337381 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:18.337381 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:18.337381 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:18.338256 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:18.338256 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:18.338256 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:18.338256 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:18.338256 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:18.338256 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:18.338256 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:18.338256 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:18.338256 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:18.338256 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:18.338256 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:18.338256 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:18.338256 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:18.338256 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:18.338256 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:18.338256 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:18.338256 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:18.339245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:18.339245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:18.339245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:18.339245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:18.341596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 346/691] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:18.347775 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164145a4-7fce-4a68-9967-4b83be621067 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:18.389149 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2295da-f0b9-45fc-a39d-9c5a447aa272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:18.389845 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2295da-f0b9-45fc-a39d-9c5a447aa272 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:18.390103 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2295da-f0b9-45fc-a39d-9c5a447aa272 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:18.390375 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2295da-f0b9-45fc-a39d-9c5a447aa272 None admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:18.391105 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2295da-f0b9-45fc-a39d-9c5a447aa272 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:18.391435 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2295da-f0b9-45fc-a39d-9c5a447aa272 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:18.392811 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2295da-f0b9-45fc-a39d-9c5a447aa272 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:18.424240 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 346/692] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:44:18.425199 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2295da-f0b9-45fc-a39d-9c5a447aa272 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:18.501431 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:18.505777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:18.523387 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164145a4-7fce-4a68-9967-4b83be621067 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:18.558393 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd41254-feee-471b-b260-18a15aa4aa26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:18.559382 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd41254-feee-471b-b260-18a15aa4aa26 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:18.559844 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd41254-feee-471b-b260-18a15aa4aa26 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:18.560323 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd41254-feee-471b-b260-18a15aa4aa26 None admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:18.561221 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd41254-feee-471b-b260-18a15aa4aa26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:18.561723 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd41254-feee-471b-b260-18a15aa4aa26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:18.563293 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd41254-feee-471b-b260-18a15aa4aa26 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:18.566624 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2295da-f0b9-45fc-a39d-9c5a447aa272 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:18.576201 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 347/693] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:44:18.577677 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7682a87-0e9e-47b4-8ebd-62e3f5a4f62a None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:18.578162 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7682a87-0e9e-47b4-8ebd-62e3f5a4f62a None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:18.578607 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7682a87-0e9e-47b4-8ebd-62e3f5a4f62a None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:18.580267 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 348/694] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:18.581346 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7682a87-0e9e-47b4-8ebd-62e3f5a4f62a None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:18.612143 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03861336-9234-4bbb-9e96-80b1daa05c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:18.613166 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03861336-9234-4bbb-9e96-80b1daa05c91 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:18.613446 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03861336-9234-4bbb-9e96-80b1daa05c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:18.613722 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03861336-9234-4bbb-9e96-80b1daa05c91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:18.622506 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03861336-9234-4bbb-9e96-80b1daa05c91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:18.624111 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03861336-9234-4bbb-9e96-80b1daa05c91 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:18.642851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-03861336-9234-4bbb-9e96-80b1daa05c91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:18.642851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:18.642851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:18.642851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:18.642851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:18.642851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:18.642851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:18.642851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:18.642851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:18.642851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:18.642851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:18.642851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:18.642851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 01:44:18.642851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 01:44:18.642851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 01:44:18.643827 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 01:44:18.643827 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:18.643827 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:18.643827 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:18.643827 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:18.643827 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:18.643827 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:18.643827 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:18.643827 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:18.643827 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:18.643827 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:18.643827 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:18.643827 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:18.643827 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:18.643827 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:18.643827 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:18.643827 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:18.644715 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 01:44:18.644715 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 01:44:18.644715 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 01:44:18.644715 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 01:44:18.644715 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 01:44:18.644715 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 01:44:18.644715 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:18.644715 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:18.645132 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 347/695] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:44:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:18.646609 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37292c1e-0db1-4a89-9893-7983ec0020ad None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:18.646889 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37292c1e-0db1-4a89-9893-7983ec0020ad None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:18.647324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37292c1e-0db1-4a89-9893-7983ec0020ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:18.668466 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-37292c1e-0db1-4a89-9893-7983ec0020ad None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:18.669996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-37292c1e-0db1-4a89-9893-7983ec0020ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:18.695371 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7682a87-0e9e-47b4-8ebd-62e3f5a4f62a None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:18.735512 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e1c1f-7dc9-40e0-883e-3a10b89c621c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:18.736453 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e1c1f-7dc9-40e0-883e-3a10b89c621c None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:18.736825 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e1c1f-7dc9-40e0-883e-3a10b89c621c None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:18.737197 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e1c1f-7dc9-40e0-883e-3a10b89c621c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:18.745345 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2e1c1f-7dc9-40e0-883e-3a10b89c621c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:18.747092 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2e1c1f-7dc9-40e0-883e-3a10b89c621c None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:18.750729 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 348/696] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:44:18.751713 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37292c1e-0db1-4a89-9893-7983ec0020ad None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:18.762725 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-8a2e1c1f-7dc9-40e0-883e-3a10b89c621c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 01:44:18.762725 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:18.762725 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:18.762725 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:18.762725 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:18.762725 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:18.762725 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:18.762725 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:18.762725 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:18.762725 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:18.762725 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:18.762725 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:18.762725 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 01:44:18.762725 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 01:44:18.762725 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 01:44:18.762725 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 01:44:18.764081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:18.764081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:18.764081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 01:44:18.764081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:18.764081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 01:44:18.764081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:18.764081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:18.764081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:18.764081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:18.764081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:18.764081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:18.764081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:18.764081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:18.764081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:18.764081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:18.764081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:18.764081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:18.764081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:18.766018 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:18.766018 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:18.766018 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 01:44:18.766018 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 01:44:18.766018 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 01:44:18.766018 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 01:44:18.766018 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 01:44:18.766018 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 01:44:18.766018 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 01:44:18.766018 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:18.766018 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 349/697] 10.222.0.26 () {56 vars in 1093 bytes} [Thu Jun 23 01:44:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:44:18.771562 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e1c1f-7dc9-40e0-883e-3a10b89c621c None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:18.775663 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:18.805753 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:18.874856 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37292c1e-0db1-4a89-9893-7983ec0020ad None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:18.886154 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e1c1f-7dc9-40e0-883e-3a10b89c621c None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:18.917596 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f67e14-6199-4b01-858b-605ed6e47f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:18.918429 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f67e14-6199-4b01-858b-605ed6e47f99 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:18.918806 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f67e14-6199-4b01-858b-605ed6e47f99 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:18.919090 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f67e14-6199-4b01-858b-605ed6e47f99 None admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:18.920112 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f67e14-6199-4b01-858b-605ed6e47f99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:18.920515 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f67e14-6199-4b01-858b-605ed6e47f99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:18.922269 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f67e14-6199-4b01-858b-605ed6e47f99 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:18.923023 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9567d437-1ffa-4c34-a570-7d7b0f9d87ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:18.923987 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567d437-1ffa-4c34-a570-7d7b0f9d87ab None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:18.924405 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567d437-1ffa-4c34-a570-7d7b0f9d87ab None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:18.924905 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567d437-1ffa-4c34-a570-7d7b0f9d87ab None admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:18.925934 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9567d437-1ffa-4c34-a570-7d7b0f9d87ab None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:18.926394 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9567d437-1ffa-4c34-a570-7d7b0f9d87ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:18.927993 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9567d437-1ffa-4c34-a570-7d7b0f9d87ab None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:18.938082 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 349/698] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jun 23 01:44:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:18.939829 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f67e14-6199-4b01-858b-605ed6e47f99 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:18.949286 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 350/699] 10.222.0.26 () {56 vars in 1133 bytes} [Thu Jun 23 01:44:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:18.950498 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9567d437-1ffa-4c34-a570-7d7b0f9d87ab None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:19.029689 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:19.059633 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f67e14-6199-4b01-858b-605ed6e47f99 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:19.061466 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:19.077399 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9567d437-1ffa-4c34-a570-7d7b0f9d87ab None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:19.105090 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19aca3a-627f-4f37-ac12-7237e232e940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:19.106130 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19aca3a-627f-4f37-ac12-7237e232e940 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:19.106426 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19aca3a-627f-4f37-ac12-7237e232e940 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:19.106706 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19aca3a-627f-4f37-ac12-7237e232e940 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:19.115371 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e160d-f391-4896-b5a6-09e8b5eeaa20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:19.116341 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e160d-f391-4896-b5a6-09e8b5eeaa20 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:19.116826 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e160d-f391-4896-b5a6-09e8b5eeaa20 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:19.117287 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e160d-f391-4896-b5a6-09e8b5eeaa20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:19.118419 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e160d-f391-4896-b5a6-09e8b5eeaa20 None admin] RBAC: Adding query filter params (role.id=c20e054f00164967a4c16e7500c93361, scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c, user.id=28e59688485241c4b6d51f99bbb3d059) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:19.119047 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e160d-f391-4896-b5a6-09e8b5eeaa20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:19.120077 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19aca3a-627f-4f37-ac12-7237e232e940 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:19.120948 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e160d-f391-4896-b5a6-09e8b5eeaa20 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:19.121530 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19aca3a-627f-4f37-ac12-7237e232e940 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:19.128305 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 351/700] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:44:18 2022] GET /identity/v3/role_assignments?user.id=28e59688485241c4b6d51f99bbb3d059&scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c&role.id=c20e054f00164967a4c16e7500c93361 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:19.129569 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e160d-f391-4896-b5a6-09e8b5eeaa20 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:19.143742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-a19aca3a-627f-4f37-ac12-7237e232e940 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:19.143742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:19.143742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:19.143742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:19.143742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:19.143742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:19.143742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:19.143742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:19.143742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:19.143742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:19.143742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:19.143742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:19.143742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:19.143742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:19.143742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:19.144710 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:19.144710 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:19.144710 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:19.144710 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:19.144710 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:19.144710 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:19.144710 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:19.144710 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:19.144710 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:19.144710 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:19.144710 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:19.144710 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:19.144710 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:19.144710 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:19.144710 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:19.144710 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:19.144710 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:19.145842 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:19.145842 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:19.145842 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:19.145842 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:19.145842 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 350/701] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:19.155243 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19aca3a-627f-4f37-ac12-7237e232e940 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:19.274163 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19aca3a-627f-4f37-ac12-7237e232e940 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:19.314885 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca3a25-6e79-4237-b88c-9072812e55ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:19.316360 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca3a25-6e79-4237-b88c-9072812e55ab None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:19.316669 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca3a25-6e79-4237-b88c-9072812e55ab None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:19.316958 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca3a25-6e79-4237-b88c-9072812e55ab None admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:19.317733 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ca3a25-6e79-4237-b88c-9072812e55ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:19.318050 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ca3a25-6e79-4237-b88c-9072812e55ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:19.319543 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ca3a25-6e79-4237-b88c-9072812e55ab None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:19.336353 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 351/702] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:19.348468 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca3a25-6e79-4237-b88c-9072812e55ab None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:19.352596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e160d-f391-4896-b5a6-09e8b5eeaa20 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:19.381489 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:19.385660 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:19.396469 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f2877-bfba-469f-9e11-a27033fffa80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:19.397121 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f2877-bfba-469f-9e11-a27033fffa80 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:19.397399 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f2877-bfba-469f-9e11-a27033fffa80 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:19.397670 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f2877-bfba-469f-9e11-a27033fffa80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:19.470259 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341f2877-bfba-469f-9e11-a27033fffa80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:19.471750 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341f2877-bfba-469f-9e11-a27033fffa80 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:19.478756 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca3a25-6e79-4237-b88c-9072812e55ab None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:19.516934 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f42d03-ac6c-43e9-9a07-b7a6c36d8587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:19.517622 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f42d03-ac6c-43e9-9a07-b7a6c36d8587 None admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:19.517906 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f42d03-ac6c-43e9-9a07-b7a6c36d8587 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:19.518186 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f42d03-ac6c-43e9-9a07-b7a6c36d8587 None admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:19.519108 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f42d03-ac6c-43e9-9a07-b7a6c36d8587 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:19.521318 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f42d03-ac6c-43e9-9a07-b7a6c36d8587 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:19.546138 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 352/703] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Jun 23 01:44:19 2022] POST /identity/v3/users => generated 254 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:44:19.576659 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 352/704] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:44:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 01:44:19.582012 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f42d03-ac6c-43e9-9a07-b7a6c36d8587 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:19.660623 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef2203-0d06-445b-b587-f42fc5e4da3a None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:19.660958 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef2203-0d06-445b-b587-f42fc5e4da3a None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:19.661174 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef2203-0d06-445b-b587-f42fc5e4da3a None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:19.662691 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 353/705] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 01:44:19.667108 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:19.675054 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914569cd-9014-4745-99ce-5b79b1f5a9f8 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:19.675699 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914569cd-9014-4745-99ce-5b79b1f5a9f8 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:19.676537 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914569cd-9014-4745-99ce-5b79b1f5a9f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:19.701022 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-914569cd-9014-4745-99ce-5b79b1f5a9f8 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:19.703399 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-914569cd-9014-4745-99ce-5b79b1f5a9f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:19.704528 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f42d03-ac6c-43e9-9a07-b7a6c36d8587 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:19.732866 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:19.748373 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b537623-f2b0-44ff-93c8-99b4e2e46f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:19.749185 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b537623-f2b0-44ff-93c8-99b4e2e46f2b None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:19.749467 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b537623-f2b0-44ff-93c8-99b4e2e46f2b None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:19.749741 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b537623-f2b0-44ff-93c8-99b4e2e46f2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:19.762148 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b537623-f2b0-44ff-93c8-99b4e2e46f2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:19.763779 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b537623-f2b0-44ff-93c8-99b4e2e46f2b None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:19.782769 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-5b537623-f2b0-44ff-93c8-99b4e2e46f2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:19.782769 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:19.782769 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:19.782769 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:19.782769 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:19.782769 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:19.782769 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:19.782769 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:19.782769 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:19.782769 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:19.782769 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:19.782769 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:19.782769 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:19.782769 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:19.782769 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:19.783758 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:19.783758 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:19.783758 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:19.783758 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:19.783758 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:19.783758 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:19.783758 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:19.783758 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:19.783758 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:19.783758 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:19.783758 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:19.783758 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:19.783758 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:19.783758 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:19.783758 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:19.783758 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:19.783758 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:19.784923 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:19.784923 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:19.784923 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:19.784923 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:19.784923 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 353/706] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:19.790310 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b537623-f2b0-44ff-93c8-99b4e2e46f2b None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:19.794646 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 354/707] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 01:44:19.819582 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914569cd-9014-4745-99ce-5b79b1f5a9f8 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:19.907891 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b537623-f2b0-44ff-93c8-99b4e2e46f2b None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:19.940018 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:19.943205 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914569cd-9014-4745-99ce-5b79b1f5a9f8 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:19.953840 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8015b803-6d01-478f-9461-dce309c40c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:19.954544 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8015b803-6d01-478f-9461-dce309c40c45 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:19.955389 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8015b803-6d01-478f-9461-dce309c40c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:19.955743 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8015b803-6d01-478f-9461-dce309c40c45 None admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:19.956525 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8015b803-6d01-478f-9461-dce309c40c45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:19.957306 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8015b803-6d01-478f-9461-dce309c40c45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:19.959144 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8015b803-6d01-478f-9461-dce309c40c45 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:19.975011 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 354/708] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:44:19.980641 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8015b803-6d01-478f-9461-dce309c40c45 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:19.984725 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d076f48-e4fa-46bc-8a41-6f0755f89ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:19.985537 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d076f48-e4fa-46bc-8a41-6f0755f89ed3 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:19.985818 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d076f48-e4fa-46bc-8a41-6f0755f89ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:19.986091 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d076f48-e4fa-46bc-8a41-6f0755f89ed3 None admin] PATH_INFO: `/v3/services/network` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:19.986757 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d076f48-e4fa-46bc-8a41-6f0755f89ed3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:19.988219 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d076f48-e4fa-46bc-8a41-6f0755f89ed3 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:20.004505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-7d076f48-e4fa-46bc-8a41-6f0755f89ed3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 01:44:20.004505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:20.004505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 01:44:20.004505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 01:44:20.004505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 01:44:20.004505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 01:44:20.004505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 01:44:20.004505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:44:20.004505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 01:44:20.004505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:20.004505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 01:44:20.004505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:20.004505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:20.004505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:20.004505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:20.004505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:20.004505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:20.005568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:20.005568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:20.005568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:20.005568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:20.005568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:20.005568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:20.005568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 01:44:20.005568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 01:44:20.005568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 01:44:20.005568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 01:44:20.005568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:20.005568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:20.005568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:20.005568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:20.005568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:20.005568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:20.006392 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:20.006392 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:20.006392 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:20.006392 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:20.006392 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:20.006392 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:20.006392 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:20.006392 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:20.006392 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:20.006392 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:20.006392 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 01:44:20.006392 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:44:20.006392 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 01:44:20.006392 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:20.006392 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 355/709] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:44:19 2022] GET /identity/v3/services/network => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:20.012395 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d076f48-e4fa-46bc-8a41-6f0755f89ed3 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:20.037611 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:20.094102 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8015b803-6d01-478f-9461-dce309c40c45 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:20.125565 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d076f48-e4fa-46bc-8a41-6f0755f89ed3 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:20.129458 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d100e2-f138-4d2c-b5b7-f5af3b3709db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:20.130266 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d100e2-f138-4d2c-b5b7-f5af3b3709db None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:20.130540 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d100e2-f138-4d2c-b5b7-f5af3b3709db None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:20.130814 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d100e2-f138-4d2c-b5b7-f5af3b3709db None admin] PATH_INFO: `/v3/projects/service` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:20.138085 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d100e2-f138-4d2c-b5b7-f5af3b3709db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:20.139653 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d100e2-f138-4d2c-b5b7-f5af3b3709db None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:20.153929 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-e7d100e2-f138-4d2c-b5b7-f5af3b3709db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:20.153929 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:20.153929 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:20.153929 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:20.153929 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:20.153929 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:20.153929 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:20.153929 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:20.153929 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:20.153929 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:20.153929 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:20.153929 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:20.153929 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 01:44:20.153929 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 01:44:20.153929 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 01:44:20.155856 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 01:44:20.155856 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:20.155856 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:20.155856 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:20.155856 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:20.155856 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:20.155856 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:20.155856 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:20.155856 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:20.155856 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:20.155856 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:20.155856 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:20.155856 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:20.155856 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:20.155856 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:20.155856 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:20.155856 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:20.157802 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 01:44:20.157802 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 01:44:20.157802 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 01:44:20.157802 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 01:44:20.157802 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 01:44:20.157802 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 01:44:20.157802 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:20.157802 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:20.163401 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 355/710] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:44:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:44:20.163401 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d100e2-f138-4d2c-b5b7-f5af3b3709db None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:20.165733 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875d2113-109b-4ad0-b6e6-10809bff8646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:20.166402 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d2113-109b-4ad0-b6e6-10809bff8646 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:20.166685 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d2113-109b-4ad0-b6e6-10809bff8646 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:20.166960 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d2113-109b-4ad0-b6e6-10809bff8646 None admin] PATH_INFO: `/v3/services` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:20.167937 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875d2113-109b-4ad0-b6e6-10809bff8646 None admin] RBAC: Adding query filter params (name=network) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:20.168247 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875d2113-109b-4ad0-b6e6-10809bff8646 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:20.169772 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875d2113-109b-4ad0-b6e6-10809bff8646 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:20.174853 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 356/711] 10.222.0.26 () {56 vars in 1112 bytes} [Thu Jun 23 01:44:20 2022] GET /identity/v3/services?name=network => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:44:20.180835 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875d2113-109b-4ad0-b6e6-10809bff8646 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:20.271238 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:20.273892 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:20.289106 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875d2113-109b-4ad0-b6e6-10809bff8646 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:20.358422 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3657b0d-ab9d-464d-ba4f-c616e5e0adbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:20.359183 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3657b0d-ab9d-464d-ba4f-c616e5e0adbf None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:20.359401 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3657b0d-ab9d-464d-ba4f-c616e5e0adbf None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:20.359678 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3657b0d-ab9d-464d-ba4f-c616e5e0adbf None admin] PATH_INFO: `/v3/services` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:20.360408 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3657b0d-ab9d-464d-ba4f-c616e5e0adbf None admin] RBAC: Adding query filter params (type=network) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:20.360712 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3657b0d-ab9d-464d-ba4f-c616e5e0adbf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:20.362120 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3657b0d-ab9d-464d-ba4f-c616e5e0adbf None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:20.371489 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d100e2-f138-4d2c-b5b7-f5af3b3709db None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:20.384033 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 357/712] 10.222.0.26 () {56 vars in 1112 bytes} [Thu Jun 23 01:44:20 2022] GET /identity/v3/services?type=network => generated 124 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:20.401791 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f42b9-2527-4e76-9609-952f8e46c49c None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:20.402152 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f42b9-2527-4e76-9609-952f8e46c49c None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:20.402354 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f42b9-2527-4e76-9609-952f8e46c49c None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:20.404156 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 358/713] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 01:44:20.418547 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5daa9a-176f-4a0d-81df-0379a22d8190 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:20.418895 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5daa9a-176f-4a0d-81df-0379a22d8190 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:20.419855 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5daa9a-176f-4a0d-81df-0379a22d8190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:20.445114 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba466bf-17c9-443e-a62f-c2e05e5fc445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:20.446108 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba466bf-17c9-443e-a62f-c2e05e5fc445 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:20.446528 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba466bf-17c9-443e-a62f-c2e05e5fc445 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:20.446960 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba466bf-17c9-443e-a62f-c2e05e5fc445 None admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:20.447997 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba466bf-17c9-443e-a62f-c2e05e5fc445 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:20.448448 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba466bf-17c9-443e-a62f-c2e05e5fc445 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:20.450426 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba466bf-17c9-443e-a62f-c2e05e5fc445 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:20.457034 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-cf5daa9a-176f-4a0d-81df-0379a22d8190 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:20.462299 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-cf5daa9a-176f-4a0d-81df-0379a22d8190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:20.475101 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 356/714] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jun 23 01:44:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:44:20.481019 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba466bf-17c9-443e-a62f-c2e05e5fc445 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:20.577603 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 359/715] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:44:20.622318 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5daa9a-176f-4a0d-81df-0379a22d8190 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:20.633838 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba466bf-17c9-443e-a62f-c2e05e5fc445 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:20.634789 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:20.676868 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eb08df-771e-4e07-8c73-463f24fbf54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:20.677546 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eb08df-771e-4e07-8c73-463f24fbf54e None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:20.677825 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eb08df-771e-4e07-8c73-463f24fbf54e None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:20.678097 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eb08df-771e-4e07-8c73-463f24fbf54e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:20.678942 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eb08df-771e-4e07-8c73-463f24fbf54e None admin] RBAC: Adding query filter params (role.id=7b20d65f49ee4a48894b6c060bb5d580, scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c, user.id=ae4e83df33b64a3ebf28f54e2d94fe63) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:20.682797 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eb08df-771e-4e07-8c73-463f24fbf54e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:20.682797 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eb08df-771e-4e07-8c73-463f24fbf54e None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:20.689319 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 357/716] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:44:20 2022] GET /identity/v3/role_assignments?user.id=ae4e83df33b64a3ebf28f54e2d94fe63&scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c&role.id=7b20d65f49ee4a48894b6c060bb5d580 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:44:20.724777 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:20.754938 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5daa9a-176f-4a0d-81df-0379a22d8190 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:20.817003 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0715ea03-5fc7-4370-9085-7f13a25270d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:20.817838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0715ea03-5fc7-4370-9085-7f13a25270d3 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:20.818119 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0715ea03-5fc7-4370-9085-7f13a25270d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:20.818389 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0715ea03-5fc7-4370-9085-7f13a25270d3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:20.819058 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0715ea03-5fc7-4370-9085-7f13a25270d3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:20.820478 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0715ea03-5fc7-4370-9085-7f13a25270d3 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:20.836021 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-0715ea03-5fc7-4370-9085-7f13a25270d3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 01:44:20.836021 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:20.836021 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 01:44:20.836021 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 01:44:20.836021 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 01:44:20.836021 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 01:44:20.836021 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 01:44:20.836021 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:44:20.836021 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 01:44:20.836021 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:20.836021 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 01:44:20.836021 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:20.836021 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:20.836021 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:20.836021 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:20.836021 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:20.837267 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:20.837267 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:20.837267 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:20.837267 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:20.837267 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:20.837267 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:20.837267 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:20.837267 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 01:44:20.837267 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 01:44:20.837267 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 01:44:20.837267 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 01:44:20.837267 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:20.837267 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:20.837267 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:20.837267 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:20.837267 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:20.838124 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:20.838124 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:20.838124 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:20.838124 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:20.838124 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:20.838124 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:20.838124 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:20.838124 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:20.838124 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:20.838124 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:20.838124 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:20.838124 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 01:44:20.838124 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:44:20.838124 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 01:44:20.838124 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:20.838124 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 360/717] 10.222.0.26 () {56 vars in 1129 bytes} [Thu Jun 23 01:44:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:44:20.843594 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eb08df-771e-4e07-8c73-463f24fbf54e None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:20.983324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eb08df-771e-4e07-8c73-463f24fbf54e None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:21.025561 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2482e9-46ea-4c7a-af21-76547e930836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:21.026238 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2482e9-46ea-4c7a-af21-76547e930836 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:21.026518 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2482e9-46ea-4c7a-af21-76547e930836 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:21.026790 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2482e9-46ea-4c7a-af21-76547e930836 None admin] PATH_INFO: `/v3/services` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:21.027537 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2482e9-46ea-4c7a-af21-76547e930836 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:21.027843 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2482e9-46ea-4c7a-af21-76547e930836 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:21.029226 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2482e9-46ea-4c7a-af21-76547e930836 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:21.038709 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 358/718] 10.222.0.26 () {56 vars in 1124 bytes} [Thu Jun 23 01:44:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:21.044325 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0715ea03-5fc7-4370-9085-7f13a25270d3 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:21.178150 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0715ea03-5fc7-4370-9085-7f13a25270d3 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:21.223506 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3f5fae-b789-4cbf-9ba7-b8641889429d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:21.224583 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f5fae-b789-4cbf-9ba7-b8641889429d None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:21.225058 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f5fae-b789-4cbf-9ba7-b8641889429d None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:21.225534 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f5fae-b789-4cbf-9ba7-b8641889429d None admin] PATH_INFO: `/v3/services` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:21.226440 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3f5fae-b789-4cbf-9ba7-b8641889429d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:21.226931 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3f5fae-b789-4cbf-9ba7-b8641889429d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:21.228562 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3f5fae-b789-4cbf-9ba7-b8641889429d None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:21.234055 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 361/719] 10.222.0.26 () {56 vars in 1124 bytes} [Thu Jun 23 01:44:21 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:21.827437 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94381f-095d-4299-a76c-6c78b051db62 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:21.827964 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94381f-095d-4299-a76c-6c78b051db62 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:21.827964 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94381f-095d-4299-a76c-6c78b051db62 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:21.829595 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 359/720] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:21.835134 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e7da97-6f22-4e88-8696-bdaaf6bcd0a2 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:21.835563 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e7da97-6f22-4e88-8696-bdaaf6bcd0a2 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:21.835781 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e7da97-6f22-4e88-8696-bdaaf6bcd0a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:21.847882 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:21.853152 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-36e7da97-6f22-4e88-8696-bdaaf6bcd0a2 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:21.854364 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-36e7da97-6f22-4e88-8696-bdaaf6bcd0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:21.921491 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 362/721] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 01:44:21.945235 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94381f-095d-4299-a76c-6c78b051db62 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:21.977459 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:22.099853 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94381f-095d-4299-a76c-6c78b051db62 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:22.144116 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d3861-4227-4219-aa6a-81eb080a68d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:22.144943 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d3861-4227-4219-aa6a-81eb080a68d5 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:22.145227 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d3861-4227-4219-aa6a-81eb080a68d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:22.145502 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d3861-4227-4219-aa6a-81eb080a68d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:22.165977 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9d3861-4227-4219-aa6a-81eb080a68d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:44:22.173646 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-af9d3861-4227-4219-aa6a-81eb080a68d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:44:22.173646 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:22.173646 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:22.173646 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:22.173646 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:22.173646 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:22.173646 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:22.173646 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:22.173646 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:22.173646 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:22.173646 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:22.173646 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:22.173646 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:44:22.173646 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:44:22.173646 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:44:22.173646 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:44:22.174539 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:44:22.174539 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:44:22.174539 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:44:22.174539 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:44:22.174539 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:22.174539 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:22.174539 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:22.174539 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:22.174539 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:22.174539 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:22.174539 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:22.174539 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:22.174539 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:22.174539 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:22.174539 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:22.174539 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:22.175441 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:22.175441 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:22.175441 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:22.175441 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:22.175441 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:44:22.175441 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:44:22.175441 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:44:22.175441 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:44:22.175441 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:44:22.175441 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:44:22.175441 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:44:22.175441 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:22.175441 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 360/722] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jun 23 01:44:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:22.180397 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2627d9d7-0c03-456a-8689-3c06ef8c63a3 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:22.180681 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2627d9d7-0c03-456a-8689-3c06ef8c63a3 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:22.180958 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2627d9d7-0c03-456a-8689-3c06ef8c63a3 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:22.182328 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 363/723] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:22.184968 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d3861-4227-4219-aa6a-81eb080a68d5 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:22.187650 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a112c4-7d18-4585-a5c2-3f59df25f3ab None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:22.187942 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a112c4-7d18-4585-a5c2-3f59df25f3ab None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:22.188212 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a112c4-7d18-4585-a5c2-3f59df25f3ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:22.213193 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-45a112c4-7d18-4585-a5c2-3f59df25f3ab None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:22.214747 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-45a112c4-7d18-4585-a5c2-3f59df25f3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:22.328342 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 364/724] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:44:22.339715 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:22.352258 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d3861-4227-4219-aa6a-81eb080a68d5 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:22.365480 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a112c4-7d18-4585-a5c2-3f59df25f3ab None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:22.417012 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38798f66-34cd-403f-8eed-3bf24c3d2853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:22.418028 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38798f66-34cd-403f-8eed-3bf24c3d2853 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:22.418537 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38798f66-34cd-403f-8eed-3bf24c3d2853 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:22.419006 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38798f66-34cd-403f-8eed-3bf24c3d2853 None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:22.420752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38798f66-34cd-403f-8eed-3bf24c3d2853 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:22.426518 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38798f66-34cd-403f-8eed-3bf24c3d2853 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:22.428268 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38798f66-34cd-403f-8eed-3bf24c3d2853 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:22.441630 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 361/725] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:44:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:22.450193 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38798f66-34cd-403f-8eed-3bf24c3d2853 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:22.478746 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:22.536694 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a112c4-7d18-4585-a5c2-3f59df25f3ab None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:22.600821 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd84ceb-b422-4aec-8c01-42b4dbdfb226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:22.602007 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd84ceb-b422-4aec-8c01-42b4dbdfb226 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:22.602728 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd84ceb-b422-4aec-8c01-42b4dbdfb226 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:22.604140 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd84ceb-b422-4aec-8c01-42b4dbdfb226 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:22.624144 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd84ceb-b422-4aec-8c01-42b4dbdfb226 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:44:22.632881 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-cfd84ceb-b422-4aec-8c01-42b4dbdfb226 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 01:44:22.632881 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:22.632881 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:22.632881 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:22.632881 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:22.632881 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:22.632881 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:22.632881 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:22.632881 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:22.632881 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:22.632881 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:22.632881 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:22.632881 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:44:22.632881 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:44:22.632881 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:44:22.632881 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:44:22.633899 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:44:22.633899 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:44:22.633899 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:44:22.633899 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:44:22.633899 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:22.633899 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:22.633899 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:22.633899 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:22.633899 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:22.633899 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:22.633899 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:22.633899 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:22.633899 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:22.633899 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:22.633899 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:22.633899 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:22.634739 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:22.634739 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:22.634739 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:22.634739 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:22.634739 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:44:22.634739 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:44:22.634739 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:44:22.634739 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:44:22.634739 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:44:22.634739 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:44:22.634739 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 01:44:22.634739 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:22.637674 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 365/726] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jun 23 01:44:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:44:22.643323 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38798f66-34cd-403f-8eed-3bf24c3d2853 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:22.644235 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd84ceb-b422-4aec-8c01-42b4dbdfb226 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:22.710094 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb16fdda-e8f8-4642-8623-679c1cfed26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:22.711399 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16fdda-e8f8-4642-8623-679c1cfed26d None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:22.711878 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16fdda-e8f8-4642-8623-679c1cfed26d None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:22.712376 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16fdda-e8f8-4642-8623-679c1cfed26d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:22.733854 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb16fdda-e8f8-4642-8623-679c1cfed26d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:22.735680 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb16fdda-e8f8-4642-8623-679c1cfed26d None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:22.768232 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-bb16fdda-e8f8-4642-8623-679c1cfed26d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:22.768232 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:22.768232 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:22.768232 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:22.768232 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:22.768232 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:22.768232 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:22.768232 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:22.768232 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:22.768232 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:22.768232 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:22.768232 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:22.768232 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:22.768232 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:22.768232 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:22.769208 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:22.769208 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:22.769208 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:22.769208 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:22.769208 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:22.769208 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:22.769208 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:22.769208 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:22.769208 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:22.769208 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:22.769208 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:22.769208 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:22.769208 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:22.769208 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:22.769208 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:22.769208 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:22.769208 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:22.770094 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:22.770094 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:22.770094 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:22.770094 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:22.772794 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 362/727] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:44:22.779254 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb16fdda-e8f8-4642-8623-679c1cfed26d None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:22.787174 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:22.799644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd84ceb-b422-4aec-8c01-42b4dbdfb226 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:22.847553 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:22.879998 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f637d8af-40e8-4d64-8b34-57dee65a445d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:22.880591 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f637d8af-40e8-4d64-8b34-57dee65a445d None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:22.880869 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f637d8af-40e8-4d64-8b34-57dee65a445d None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:22.881143 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f637d8af-40e8-4d64-8b34-57dee65a445d None admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:22.881900 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f637d8af-40e8-4d64-8b34-57dee65a445d None admin] RBAC: Adding query filter params (name=service) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:22.882201 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f637d8af-40e8-4d64-8b34-57dee65a445d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:22.883628 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f637d8af-40e8-4d64-8b34-57dee65a445d None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:22.913536 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 366/728] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Jun 23 01:44:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:22.919292 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb16fdda-e8f8-4642-8623-679c1cfed26d None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:22.920804 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f637d8af-40e8-4d64-8b34-57dee65a445d None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:22.991937 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01266e01-8e32-4fa0-8f82-cf632fd402fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:22.992633 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01266e01-8e32-4fa0-8f82-cf632fd402fa None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:22.992919 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01266e01-8e32-4fa0-8f82-cf632fd402fa None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:22.993191 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01266e01-8e32-4fa0-8f82-cf632fd402fa None admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:22.993933 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01266e01-8e32-4fa0-8f82-cf632fd402fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:22.994241 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01266e01-8e32-4fa0-8f82-cf632fd402fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:22.995670 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01266e01-8e32-4fa0-8f82-cf632fd402fa None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:23.018926 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 363/729] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:23.024289 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01266e01-8e32-4fa0-8f82-cf632fd402fa None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:23.059798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f637d8af-40e8-4d64-8b34-57dee65a445d None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:23.096713 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bc6077-f8f3-4207-bd15-0e055b2136bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:23.097832 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bc6077-f8f3-4207-bd15-0e055b2136bf None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:23.098287 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bc6077-f8f3-4207-bd15-0e055b2136bf None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:23.098760 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bc6077-f8f3-4207-bd15-0e055b2136bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:23.110345 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bc6077-f8f3-4207-bd15-0e055b2136bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:23.112079 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bc6077-f8f3-4207-bd15-0e055b2136bf None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:23.126903 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:23.130445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-04bc6077-f8f3-4207-bd15-0e055b2136bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:23.130445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:23.130445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:23.130445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:23.130445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:23.130445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:23.130445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:23.130445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:23.130445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:23.130445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:23.130445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:23.130445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:23.130445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:23.130445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:23.130445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:23.131396 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:23.131396 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:23.131396 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:23.131396 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:23.131396 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:23.131396 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:23.131396 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:23.131396 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:23.131396 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:23.131396 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:23.131396 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:23.131396 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:23.131396 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:23.131396 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:23.131396 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:23.131396 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:23.131396 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:23.132293 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:23.132293 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:23.132293 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:23.132293 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:23.134600 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 367/730] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:23.140566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ce12c9-2249-4b56-9793-3c7a5ea41c57 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:23.141053 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ce12c9-2249-4b56-9793-3c7a5ea41c57 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:23.141504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ce12c9-2249-4b56-9793-3c7a5ea41c57 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:23.143249 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 368/731] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:23.144178 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ce12c9-2249-4b56-9793-3c7a5ea41c57 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:23.174820 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01266e01-8e32-4fa0-8f82-cf632fd402fa None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:23.191473 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:23.210685 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c14d33-ef9e-487f-8029-d15a6c71551e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:23.211846 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c14d33-ef9e-487f-8029-d15a6c71551e None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:23.212355 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c14d33-ef9e-487f-8029-d15a6c71551e None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:23.212630 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c14d33-ef9e-487f-8029-d15a6c71551e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:23.219792 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c14d33-ef9e-487f-8029-d15a6c71551e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:23.221259 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c14d33-ef9e-487f-8029-d15a6c71551e None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:23.235653 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-19c14d33-ef9e-487f-8029-d15a6c71551e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 01:44:23.235653 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:23.235653 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:23.235653 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:23.235653 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:23.235653 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:23.235653 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:23.235653 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:23.235653 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:23.235653 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:23.235653 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:23.235653 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:23.235653 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 01:44:23.235653 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 01:44:23.235653 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 01:44:23.235653 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 01:44:23.236605 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:23.236605 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:23.236605 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 01:44:23.236605 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:23.236605 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 01:44:23.236605 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:23.236605 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:23.236605 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:23.236605 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:23.236605 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:23.236605 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:23.236605 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:23.236605 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:23.236605 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:23.236605 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:23.236605 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:23.236605 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:23.236605 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:23.237604 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:23.237604 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:23.237604 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 01:44:23.237604 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 01:44:23.237604 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 01:44:23.237604 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 01:44:23.237604 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 01:44:23.237604 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 01:44:23.237604 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 01:44:23.237604 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:23.241597 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 364/732] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jun 23 01:44:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:44:23.242721 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4cb3bc-4c0e-487c-a2e5-27c0bdce41c1 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:23.243008 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4cb3bc-4c0e-487c-a2e5-27c0bdce41c1 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:23.243311 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4cb3bc-4c0e-487c-a2e5-27c0bdce41c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:23.258505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ce12c9-2249-4b56-9793-3c7a5ea41c57 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:23.259310 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-8e4cb3bc-4c0e-487c-a2e5-27c0bdce41c1 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:23.260490 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-8e4cb3bc-4c0e-487c-a2e5-27c0bdce41c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:23.303030 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f636d7a3-43e5-479b-b88d-339e0297d662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:23.303755 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636d7a3-43e5-479b-b88d-339e0297d662 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:23.304088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636d7a3-43e5-479b-b88d-339e0297d662 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:23.304361 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f636d7a3-43e5-479b-b88d-339e0297d662 None admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:23.305436 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f636d7a3-43e5-479b-b88d-339e0297d662 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:23.305740 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f636d7a3-43e5-479b-b88d-339e0297d662 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:23.307114 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f636d7a3-43e5-479b-b88d-339e0297d662 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:23.328902 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 369/733] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:23.328902 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37ad57-cd2f-4018-a4d5-e0074dde5fd7 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:23.328902 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37ad57-cd2f-4018-a4d5-e0074dde5fd7 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:23.328902 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37ad57-cd2f-4018-a4d5-e0074dde5fd7 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:23.328902 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 370/734] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:23.328902 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d37ad57-cd2f-4018-a4d5-e0074dde5fd7 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:23.334508 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 365/735] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:44:23.342666 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0714a9-58fd-49d0-bcf9-a7169153da34 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:23.342956 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0714a9-58fd-49d0-bcf9-a7169153da34 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:23.343249 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0714a9-58fd-49d0-bcf9-a7169153da34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:23.368145 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-fd0714a9-58fd-49d0-bcf9-a7169153da34 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:23.369332 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-fd0714a9-58fd-49d0-bcf9-a7169153da34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:23.418204 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:23.436452 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 366/736] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:44:23.443324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0714a9-58fd-49d0-bcf9-a7169153da34 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:23.444690 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:23.453959 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d37ad57-cd2f-4018-a4d5-e0074dde5fd7 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:23.491880 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7154b9e-d13c-4de3-b409-c727696f2360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:23.492644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7154b9e-d13c-4de3-b409-c727696f2360 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:23.492931 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7154b9e-d13c-4de3-b409-c727696f2360 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:23.493218 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7154b9e-d13c-4de3-b409-c727696f2360 None admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:23.494053 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7154b9e-d13c-4de3-b409-c727696f2360 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:23.494361 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7154b9e-d13c-4de3-b409-c727696f2360 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:23.496079 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7154b9e-d13c-4de3-b409-c727696f2360 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:23.519400 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 371/737] 10.222.0.26 () {56 vars in 1135 bytes} [Thu Jun 23 01:44:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:23.519400 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7154b9e-d13c-4de3-b409-c727696f2360 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:23.554388 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0714a9-58fd-49d0-bcf9-a7169153da34 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:23.592352 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3aa638-4f3c-4b26-9eb3-9a3777b7eaac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:23.593943 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3aa638-4f3c-4b26-9eb3-9a3777b7eaac None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:23.594231 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3aa638-4f3c-4b26-9eb3-9a3777b7eaac None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:23.594510 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3aa638-4f3c-4b26-9eb3-9a3777b7eaac None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:23.603663 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3aa638-4f3c-4b26-9eb3-9a3777b7eaac None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:23.605746 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3aa638-4f3c-4b26-9eb3-9a3777b7eaac None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:23.621298 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7154b9e-d13c-4de3-b409-c727696f2360 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:23.625772 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-1f3aa638-4f3c-4b26-9eb3-9a3777b7eaac None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 01:44:23.625772 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:23.625772 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:23.625772 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:23.625772 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:23.625772 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:23.625772 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:23.625772 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:23.625772 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:23.625772 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:23.625772 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:23.625772 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:23.625772 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 01:44:23.625772 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 01:44:23.625772 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 01:44:23.625772 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 01:44:23.626687 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:23.626687 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:23.626687 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 01:44:23.626687 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:23.626687 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 01:44:23.626687 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:23.626687 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:23.626687 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:23.626687 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:23.626687 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:23.626687 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:23.626687 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:23.626687 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:23.626687 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:23.626687 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:23.626687 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:23.626687 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:23.626687 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:23.628447 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:23.628447 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:23.628447 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 01:44:23.628447 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 01:44:23.628447 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 01:44:23.628447 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 01:44:23.628447 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 01:44:23.628447 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 01:44:23.628447 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 01:44:23.628447 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:23.628447 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 367/738] 10.222.0.26 () {56 vars in 1117 bytes} [Thu Jun 23 01:44:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:23.630096 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3aa638-4f3c-4b26-9eb3-9a3777b7eaac None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:23.660112 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefd9cf2-453b-4591-9f38-e66e55091d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:23.660885 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefd9cf2-453b-4591-9f38-e66e55091d23 None admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:23.661167 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefd9cf2-453b-4591-9f38-e66e55091d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:23.661735 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefd9cf2-453b-4591-9f38-e66e55091d23 None admin] PATH_INFO: `/v3/services` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:23.662308 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefd9cf2-453b-4591-9f38-e66e55091d23 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:23.663675 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefd9cf2-453b-4591-9f38-e66e55091d23 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:23.678181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:23.680246 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 372/739] 10.222.0.26 () {60 vars in 1141 bytes} [Thu Jun 23 01:44:23 2022] POST /identity/v3/services => generated 240 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:44:23.681217 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefd9cf2-453b-4591-9f38-e66e55091d23 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:23.705882 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:23.767924 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3aa638-4f3c-4b26-9eb3-9a3777b7eaac None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:23.822773 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568f7dae-3877-4af6-9a3f-868916a5919b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:23.824044 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f7dae-3877-4af6-9a3f-868916a5919b None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:23.824162 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f7dae-3877-4af6-9a3f-868916a5919b None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:23.824349 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f7dae-3877-4af6-9a3f-868916a5919b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:23.828858 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefd9cf2-453b-4591-9f38-e66e55091d23 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:23.837500 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568f7dae-3877-4af6-9a3f-868916a5919b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:44:23.846194 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-568f7dae-3877-4af6-9a3f-868916a5919b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:44:23.846194 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:23.846194 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:23.846194 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:23.846194 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:23.846194 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:23.846194 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:23.846194 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:23.846194 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:23.846194 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:23.846194 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:23.846194 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:23.846194 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:44:23.846194 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:44:23.846194 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:44:23.846194 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:44:23.847247 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:44:23.847247 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:44:23.847247 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:44:23.847247 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:44:23.847247 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:23.847247 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:23.847247 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:23.847247 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:23.847247 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:23.847247 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:23.847247 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:23.847247 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:23.847247 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:23.847247 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:23.847247 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:23.847247 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:23.848100 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:23.848100 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:23.848100 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:23.848100 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:23.848100 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:44:23.848100 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:44:23.848100 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:44:23.848100 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:44:23.848100 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:44:23.848100 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:44:23.848100 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:44:23.848100 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:23.848750 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 368/740] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jun 23 01:44:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:23.849864 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568f7dae-3877-4af6-9a3f-868916a5919b None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:23.882745 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e063f91d-2dd7-4925-8b6f-bea649887b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:23.883445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e063f91d-2dd7-4925-8b6f-bea649887b4a None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:23.883537 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e063f91d-2dd7-4925-8b6f-bea649887b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:23.883746 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e063f91d-2dd7-4925-8b6f-bea649887b4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:23.970434 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e063f91d-2dd7-4925-8b6f-bea649887b4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:23.972468 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e063f91d-2dd7-4925-8b6f-bea649887b4a None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:23.975477 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:23.980132 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568f7dae-3877-4af6-9a3f-868916a5919b None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:23.990659 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:24.018613 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9694169a-fe04-4641-99de-7a39d0109270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:24.019323 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694169a-fe04-4641-99de-7a39d0109270 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:24.019608 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694169a-fe04-4641-99de-7a39d0109270 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:24.019882 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694169a-fe04-4641-99de-7a39d0109270 None admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:24.020727 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9694169a-fe04-4641-99de-7a39d0109270 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:24.021034 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9694169a-fe04-4641-99de-7a39d0109270 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:24.022479 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9694169a-fe04-4641-99de-7a39d0109270 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:24.046038 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 369/741] 10.222.0.26 () {56 vars in 1149 bytes} [Thu Jun 23 01:44:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:24.046038 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9694169a-fe04-4641-99de-7a39d0109270 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:24.090813 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 373/742] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:44:23 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 01:44:24.090813 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b764c-4578-4e13-813b-2a396ac5295a None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:24.090813 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b764c-4578-4e13-813b-2a396ac5295a None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:24.090813 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b764c-4578-4e13-813b-2a396ac5295a None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:24.094032 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 374/743] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:24.094834 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39b764c-4578-4e13-813b-2a396ac5295a None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:24.152702 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9694169a-fe04-4641-99de-7a39d0109270 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:24.188582 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c41fc60-1e71-413e-b97a-224f195b1832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:24.189277 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41fc60-1e71-413e-b97a-224f195b1832 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:24.189550 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41fc60-1e71-413e-b97a-224f195b1832 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:24.189823 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41fc60-1e71-413e-b97a-224f195b1832 None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:24.190615 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c41fc60-1e71-413e-b97a-224f195b1832 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:24.190919 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c41fc60-1e71-413e-b97a-224f195b1832 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:24.192389 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c41fc60-1e71-413e-b97a-224f195b1832 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:24.205145 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 370/744] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:44:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:24.205145 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c41fc60-1e71-413e-b97a-224f195b1832 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:24.218933 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39b764c-4578-4e13-813b-2a396ac5295a None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:24.225349 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:24.234457 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:24.255768 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5e84a-7d84-44da-a7db-3ee3da1f0035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:24.256732 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5e84a-7d84-44da-a7db-3ee3da1f0035 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:24.257015 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5e84a-7d84-44da-a7db-3ee3da1f0035 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:24.257293 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5e84a-7d84-44da-a7db-3ee3da1f0035 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:24.313683 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c41fc60-1e71-413e-b97a-224f195b1832 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:24.332717 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d5e84a-7d84-44da-a7db-3ee3da1f0035 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:24.334208 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d5e84a-7d84-44da-a7db-3ee3da1f0035 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:24.351078 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce0f5b-8211-4af1-b0de-db8b4b5fa3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:24.352016 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce0f5b-8211-4af1-b0de-db8b4b5fa3ea None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:24.352366 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce0f5b-8211-4af1-b0de-db8b4b5fa3ea None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:24.352650 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce0f5b-8211-4af1-b0de-db8b4b5fa3ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:24.363794 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ce0f5b-8211-4af1-b0de-db8b4b5fa3ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:24.365483 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ce0f5b-8211-4af1-b0de-db8b4b5fa3ea None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:24.382996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-b9ce0f5b-8211-4af1-b0de-db8b4b5fa3ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:24.382996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:24.382996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:24.382996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:24.382996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:24.382996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:24.382996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:24.382996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:24.382996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:24.382996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:24.382996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:24.382996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:24.382996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:24.382996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:24.382996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:24.383948 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:24.383948 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:24.383948 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:24.383948 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:24.383948 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:24.383948 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:24.383948 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:24.383948 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:24.383948 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:24.383948 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:24.383948 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:24.383948 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:24.383948 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:24.383948 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:24.383948 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:24.383948 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:24.383948 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:24.384839 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:24.384839 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:24.384839 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:24.384839 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:24.384839 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 371/745] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:24.385677 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c01e6d-06c5-4556-9fcf-19f67e64487f None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:24.385777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c01e6d-06c5-4556-9fcf-19f67e64487f None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:24.386007 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c01e6d-06c5-4556-9fcf-19f67e64487f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:24.406495 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-06c01e6d-06c5-4556-9fcf-19f67e64487f None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:24.407750 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-06c01e6d-06c5-4556-9fcf-19f67e64487f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:24.452624 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 375/746] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:44:24 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 01:44:24.453872 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5e84a-7d84-44da-a7db-3ee3da1f0035 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:24.470723 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:24.491648 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 372/747] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:24 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 01:44:24.492646 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c01e6d-06c5-4556-9fcf-19f67e64487f None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:24.495337 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:24.563523 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5e84a-7d84-44da-a7db-3ee3da1f0035 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:24.599365 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5480f64-e4de-4114-9cac-298e4280101a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:24.599747 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c01e6d-06c5-4556-9fcf-19f67e64487f None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:24.600154 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5480f64-e4de-4114-9cac-298e4280101a None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:24.600436 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5480f64-e4de-4114-9cac-298e4280101a None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:24.600710 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5480f64-e4de-4114-9cac-298e4280101a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:24.612950 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5480f64-e4de-4114-9cac-298e4280101a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:24.615048 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5480f64-e4de-4114-9cac-298e4280101a None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:24.632823 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5957ca57-ef55-40cb-9965-9f52d685ce43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:24.633510 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5957ca57-ef55-40cb-9965-9f52d685ce43 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:24.633787 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5957ca57-ef55-40cb-9965-9f52d685ce43 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:24.634060 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5957ca57-ef55-40cb-9965-9f52d685ce43 None admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:24.634800 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5957ca57-ef55-40cb-9965-9f52d685ce43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:24.635082 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5957ca57-ef55-40cb-9965-9f52d685ce43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:24.635924 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-a5480f64-e4de-4114-9cac-298e4280101a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:24.635924 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:24.635924 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:24.635924 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:24.635924 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:24.635924 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:24.635924 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:24.635924 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:24.635924 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:24.635924 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:24.635924 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:24.635924 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:24.635924 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:24.635924 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:24.635924 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:24.636760 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:24.636760 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:24.636760 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:24.636760 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:24.636760 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:24.636760 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:24.636760 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:24.636760 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:24.636760 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:24.636760 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:24.636760 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:24.636760 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:24.636760 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:24.636760 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:24.636760 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:24.636760 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:24.636760 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:24.637644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:24.637644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:24.637644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:24.637644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:24.637862 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5957ca57-ef55-40cb-9965-9f52d685ce43 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:24.639559 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 376/748] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:24.640686 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5480f64-e4de-4114-9cac-298e4280101a None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:24.650047 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 373/749] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:24.650912 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5957ca57-ef55-40cb-9965-9f52d685ce43 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:24.711116 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:24.718694 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:24.753457 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5480f64-e4de-4114-9cac-298e4280101a None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:24.762212 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5957ca57-ef55-40cb-9965-9f52d685ce43 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:24.789804 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc23bbf-c95d-4057-9993-004843d3ff47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:24.790734 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc23bbf-c95d-4057-9993-004843d3ff47 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:24.791112 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc23bbf-c95d-4057-9993-004843d3ff47 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:24.791631 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc23bbf-c95d-4057-9993-004843d3ff47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:24.796326 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a65bf0-eaf8-4f21-8669-b3d3c4151ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:24.797072 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a65bf0-eaf8-4f21-8669-b3d3c4151ef3 None admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:24.797424 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a65bf0-eaf8-4f21-8669-b3d3c4151ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:24.797770 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a65bf0-eaf8-4f21-8669-b3d3c4151ef3 None admin] PATH_INFO: `/v3/services` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:24.798575 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a65bf0-eaf8-4f21-8669-b3d3c4151ef3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:24.800101 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a65bf0-eaf8-4f21-8669-b3d3c4151ef3 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:24.802524 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc23bbf-c95d-4057-9993-004843d3ff47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:24.804335 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc23bbf-c95d-4057-9993-004843d3ff47 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:24.816261 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 374/750] 10.222.0.26 () {60 vars in 1141 bytes} [Thu Jun 23 01:44:24 2022] POST /identity/v3/services => generated 251 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:44:24.824296 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-0fc23bbf-c95d-4057-9993-004843d3ff47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:24.824296 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:24.824296 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:24.824296 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:24.824296 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:24.824296 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:24.824296 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:24.824296 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:24.824296 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:24.824296 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:24.824296 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:24.824296 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:24.824296 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:24.824296 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:24.824296 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:24.825245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:24.825245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:24.825245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:24.825245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:24.825245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:24.825245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:24.825245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:24.825245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:24.825245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:24.825245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:24.825245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:24.825245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:24.825245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:24.825245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:24.825245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:24.825245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:24.825245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:24.826122 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:24.826122 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:24.826122 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:24.826122 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:24.828371 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a65bf0-eaf8-4f21-8669-b3d3c4151ef3 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:24.830557 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 377/751] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:24.839447 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc23bbf-c95d-4057-9993-004843d3ff47 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:24.962701 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a65bf0-eaf8-4f21-8669-b3d3c4151ef3 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:24.981319 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:24.982735 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc23bbf-c95d-4057-9993-004843d3ff47 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:25.000682 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0663c16d-3f29-4edb-afbf-b5c0fb488cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:25.001593 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663c16d-3f29-4edb-afbf-b5c0fb488cc6 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:25.001918 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663c16d-3f29-4edb-afbf-b5c0fb488cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:25.004529 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663c16d-3f29-4edb-afbf-b5c0fb488cc6 None admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:25.005834 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0663c16d-3f29-4edb-afbf-b5c0fb488cc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:25.008246 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0663c16d-3f29-4edb-afbf-b5c0fb488cc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:25.009926 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0663c16d-3f29-4edb-afbf-b5c0fb488cc6 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:25.027728 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 375/752] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:25.029035 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0663c16d-3f29-4edb-afbf-b5c0fb488cc6 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:25.030262 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:25.056999 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2468da31-5c96-4261-8d80-221c8f2a4736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:25.056999 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2468da31-5c96-4261-8d80-221c8f2a4736 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:25.056999 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2468da31-5c96-4261-8d80-221c8f2a4736 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:25.056999 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2468da31-5c96-4261-8d80-221c8f2a4736 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:25.068642 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2468da31-5c96-4261-8d80-221c8f2a4736 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:25.071145 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2468da31-5c96-4261-8d80-221c8f2a4736 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:25.086373 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-2468da31-5c96-4261-8d80-221c8f2a4736 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:25.086373 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:25.086373 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:25.086373 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:25.086373 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:25.086373 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:25.086373 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:25.086373 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:25.086373 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:25.086373 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:25.086373 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:25.086373 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:25.086373 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 01:44:25.086373 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 01:44:25.086373 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 01:44:25.087315 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 01:44:25.087315 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:25.087315 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:25.087315 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:25.087315 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:25.087315 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:25.087315 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:25.087315 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:25.087315 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:25.087315 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:25.087315 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:25.087315 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:25.087315 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:25.087315 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:25.087315 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:25.087315 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:25.087315 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:25.088202 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 01:44:25.088202 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 01:44:25.088202 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 01:44:25.088202 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 01:44:25.088202 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 01:44:25.088202 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 01:44:25.088202 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:25.088202 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:25.090675 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 378/753] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:44:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:25.091706 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2468da31-5c96-4261-8d80-221c8f2a4736 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:25.169744 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0663c16d-3f29-4edb-afbf-b5c0fb488cc6 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:25.235699 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7136ef2-ffd4-4c17-a4be-613b7b7502e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:25.239244 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7136ef2-ffd4-4c17-a4be-613b7b7502e2 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:25.239244 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7136ef2-ffd4-4c17-a4be-613b7b7502e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:25.239244 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7136ef2-ffd4-4c17-a4be-613b7b7502e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:25.239244 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7136ef2-ffd4-4c17-a4be-613b7b7502e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:25.239244 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7136ef2-ffd4-4c17-a4be-613b7b7502e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:25.239616 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7136ef2-ffd4-4c17-a4be-613b7b7502e2 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:25.249363 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2468da31-5c96-4261-8d80-221c8f2a4736 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:25.261853 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 376/754] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:25.263036 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7136ef2-ffd4-4c17-a4be-613b7b7502e2 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:25.323138 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f17a87-ba25-4bed-88ae-6f02b6b822f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:25.324601 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f17a87-ba25-4bed-88ae-6f02b6b822f9 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:25.326162 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f17a87-ba25-4bed-88ae-6f02b6b822f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:25.326721 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f17a87-ba25-4bed-88ae-6f02b6b822f9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:25.345978 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f17a87-ba25-4bed-88ae-6f02b6b822f9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:25.347908 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f17a87-ba25-4bed-88ae-6f02b6b822f9 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:25.360169 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:25.384504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-58f17a87-ba25-4bed-88ae-6f02b6b822f9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 01:44:25.384504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:25.384504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:25.384504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:25.384504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:25.384504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:25.384504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:25.384504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:25.384504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:25.384504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:25.384504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:25.384504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:25.384504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 01:44:25.384504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 01:44:25.384504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 01:44:25.384504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 01:44:25.385523 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:25.385523 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:25.385523 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 01:44:25.385523 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:25.385523 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 01:44:25.385523 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:25.385523 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:25.385523 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:25.385523 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:25.385523 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:25.385523 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:25.385523 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:25.385523 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:25.385523 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:25.385523 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:25.385523 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:25.385523 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:25.385523 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:25.386469 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:25.386469 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:25.386469 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 01:44:25.386469 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 01:44:25.386469 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 01:44:25.386469 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 01:44:25.386469 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 01:44:25.386469 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 01:44:25.386469 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 01:44:25.386469 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:25.392843 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 379/755] 10.222.0.26 () {56 vars in 1093 bytes} [Thu Jun 23 01:44:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:25.393992 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f17a87-ba25-4bed-88ae-6f02b6b822f9 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:25.448945 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:25.481796 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7136ef2-ffd4-4c17-a4be-613b7b7502e2 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:25.517957 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9836e486-3fe2-460f-8fa4-6459b347d993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:25.518285 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9836e486-3fe2-460f-8fa4-6459b347d993 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:25.518606 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9836e486-3fe2-460f-8fa4-6459b347d993 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:25.518887 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9836e486-3fe2-460f-8fa4-6459b347d993 None admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:25.519728 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9836e486-3fe2-460f-8fa4-6459b347d993 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:25.520041 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9836e486-3fe2-460f-8fa4-6459b347d993 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:25.521494 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9836e486-3fe2-460f-8fa4-6459b347d993 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:25.534101 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f17a87-ba25-4bed-88ae-6f02b6b822f9 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:25.538003 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 377/756] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jun 23 01:44:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:25.539038 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9836e486-3fe2-460f-8fa4-6459b347d993 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:25.575608 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c4933d-04e0-49f6-ad5c-b6f859633740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:25.576412 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c4933d-04e0-49f6-ad5c-b6f859633740 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:25.576693 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c4933d-04e0-49f6-ad5c-b6f859633740 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:25.576967 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c4933d-04e0-49f6-ad5c-b6f859633740 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:25.584148 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c4933d-04e0-49f6-ad5c-b6f859633740 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:25.585634 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c4933d-04e0-49f6-ad5c-b6f859633740 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:25.600206 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-c1c4933d-04e0-49f6-ad5c-b6f859633740 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:25.600206 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:25.600206 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:25.600206 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:25.600206 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:25.600206 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:25.600206 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:25.600206 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:25.600206 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:25.600206 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:25.600206 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:25.600206 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:25.600206 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 01:44:25.600206 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 01:44:25.600206 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 01:44:25.601102 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 01:44:25.601102 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:25.601102 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:25.601102 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:25.601102 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:25.601102 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:25.601102 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:25.601102 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:25.601102 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:25.601102 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:25.601102 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:25.601102 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:25.601102 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:25.601102 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:25.601102 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:25.601102 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:25.601102 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:25.601986 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 01:44:25.601986 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 01:44:25.601986 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 01:44:25.601986 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 01:44:25.601986 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 01:44:25.601986 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 01:44:25.601986 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:25.601986 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:25.601986 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 380/757] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:44:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:25.602467 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c4933d-04e0-49f6-ad5c-b6f859633740 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:25.660153 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:25.665641 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9836e486-3fe2-460f-8fa4-6459b347d993 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:25.704548 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c4933d-04e0-49f6-ad5c-b6f859633740 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:25.705315 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8eae8-38ac-4f55-99a0-59133ea98f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:25.706061 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8eae8-38ac-4f55-99a0-59133ea98f77 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:25.706414 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:25.706803 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8eae8-38ac-4f55-99a0-59133ea98f77 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:25.707086 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8eae8-38ac-4f55-99a0-59133ea98f77 None admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:25.708030 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a8eae8-38ac-4f55-99a0-59133ea98f77 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:25.708349 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a8eae8-38ac-4f55-99a0-59133ea98f77 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:25.709870 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a8eae8-38ac-4f55-99a0-59133ea98f77 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:25.729706 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 378/758] 10.222.0.26 () {56 vars in 1133 bytes} [Thu Jun 23 01:44:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:25.730470 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8eae8-38ac-4f55-99a0-59133ea98f77 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:25.742783 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9b3b1-0541-47d4-a7c6-27b0dfc4d26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:25.743574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9b3b1-0541-47d4-a7c6-27b0dfc4d26d None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:25.743856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9b3b1-0541-47d4-a7c6-27b0dfc4d26d None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:25.744135 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9b3b1-0541-47d4-a7c6-27b0dfc4d26d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:25.744985 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c9b3b1-0541-47d4-a7c6-27b0dfc4d26d None admin] RBAC: Adding query filter params (role.id=7b20d65f49ee4a48894b6c060bb5d580, scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c, user.id=28e59688485241c4b6d51f99bbb3d059) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:25.745296 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c9b3b1-0541-47d4-a7c6-27b0dfc4d26d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:25.746949 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c9b3b1-0541-47d4-a7c6-27b0dfc4d26d None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:25.759619 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 381/759] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:44:25 2022] GET /identity/v3/role_assignments?user.id=28e59688485241c4b6d51f99bbb3d059&scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c&role.id=7b20d65f49ee4a48894b6c060bb5d580 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:25.760749 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9b3b1-0541-47d4-a7c6-27b0dfc4d26d None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:25.842760 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8eae8-38ac-4f55-99a0-59133ea98f77 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:25.884516 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d59b7b5-728c-4bd8-9804-0a235713da31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:25.885300 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59b7b5-728c-4bd8-9804-0a235713da31 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:25.885586 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59b7b5-728c-4bd8-9804-0a235713da31 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:25.885864 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59b7b5-728c-4bd8-9804-0a235713da31 None admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:25.886673 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d59b7b5-728c-4bd8-9804-0a235713da31 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:25.886985 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d59b7b5-728c-4bd8-9804-0a235713da31 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:25.888714 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d59b7b5-728c-4bd8-9804-0a235713da31 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:25.898030 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9b3b1-0541-47d4-a7c6-27b0dfc4d26d None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:25.903945 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 379/760] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jun 23 01:44:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:25.905365 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc0caf-b380-44b0-ae5a-ff94accba393 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:25.905652 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc0caf-b380-44b0-ae5a-ff94accba393 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:25.905927 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc0caf-b380-44b0-ae5a-ff94accba393 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:25.907698 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 380/761] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:25.910807 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bc0caf-b380-44b0-ae5a-ff94accba393 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:25.932010 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:25.943745 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec34298-390c-4896-a6a2-6f147c1d786c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:25.944558 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec34298-390c-4896-a6a2-6f147c1d786c None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:25.944842 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec34298-390c-4896-a6a2-6f147c1d786c None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:25.945115 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec34298-390c-4896-a6a2-6f147c1d786c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:25.962852 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec34298-390c-4896-a6a2-6f147c1d786c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:25.964377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec34298-390c-4896-a6a2-6f147c1d786c None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:25.982627 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-3ec34298-390c-4896-a6a2-6f147c1d786c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:25.982627 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:25.982627 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:25.982627 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:25.982627 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:25.982627 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:25.982627 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:25.982627 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:25.982627 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:25.982627 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:25.982627 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:25.982627 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:25.982627 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:25.982627 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:25.982627 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:25.983994 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:25.983994 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:25.983994 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:25.983994 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:25.983994 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:25.983994 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:25.983994 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:25.983994 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:25.983994 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:25.983994 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:25.983994 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:25.983994 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:25.983994 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:25.983994 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:25.983994 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:25.983994 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:25.983994 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:25.984895 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:25.984895 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:25.984895 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:25.984895 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:25.984895 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 382/762] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:25.985604 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e5b66e-d0fe-42ba-aa38-992e6e5d7e6e None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:25.985764 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e5b66e-d0fe-42ba-aa38-992e6e5d7e6e None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:25.986042 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e5b66e-d0fe-42ba-aa38-992e6e5d7e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:26.003324 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-64e5b66e-d0fe-42ba-aa38-992e6e5d7e6e None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:26.004525 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-64e5b66e-d0fe-42ba-aa38-992e6e5d7e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:26.014635 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:26.028985 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bc0caf-b380-44b0-ae5a-ff94accba393 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:26.066937 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71814400-147f-42a2-bb51-9416f396b603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:26.067690 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71814400-147f-42a2-bb51-9416f396b603 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:26.067980 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71814400-147f-42a2-bb51-9416f396b603 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:26.068260 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71814400-147f-42a2-bb51-9416f396b603 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:26.069252 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71814400-147f-42a2-bb51-9416f396b603 None admin] RBAC: Adding query filter params (role.id=c20e054f00164967a4c16e7500c93361, scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c, user.id=168e6181ad22460990f1f7361b483364) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:26.069570 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71814400-147f-42a2-bb51-9416f396b603 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:26.071128 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71814400-147f-42a2-bb51-9416f396b603 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:26.076954 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 383/763] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:44:26.077667 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 381/764] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:44:25 2022] GET /identity/v3/role_assignments?user.id=168e6181ad22460990f1f7361b483364&scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c&role.id=c20e054f00164967a4c16e7500c93361 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:44:26.078430 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e5b66e-d0fe-42ba-aa38-992e6e5d7e6e None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:26.112024 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71814400-147f-42a2-bb51-9416f396b603 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:26.189820 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e5b66e-d0fe-42ba-aa38-992e6e5d7e6e None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:26.227333 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d81d595-9134-497d-8015-6c6594d102b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:26.228410 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d81d595-9134-497d-8015-6c6594d102b2 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:26.228893 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d81d595-9134-497d-8015-6c6594d102b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:26.229363 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d81d595-9134-497d-8015-6c6594d102b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:26.230330 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d81d595-9134-497d-8015-6c6594d102b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:26.230874 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d81d595-9134-497d-8015-6c6594d102b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:26.232565 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d81d595-9134-497d-8015-6c6594d102b2 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:26.246078 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 384/765] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:26.254642 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d81d595-9134-497d-8015-6c6594d102b2 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:26.262985 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:26.322649 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:26.385259 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71814400-147f-42a2-bb51-9416f396b603 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:26.398680 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d81d595-9134-497d-8015-6c6594d102b2 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:26.434621 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136afc3a-7508-451d-802d-27b91b13f934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:26.435865 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136afc3a-7508-451d-802d-27b91b13f934 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:26.435865 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136afc3a-7508-451d-802d-27b91b13f934 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:26.436062 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136afc3a-7508-451d-802d-27b91b13f934 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:26.447372 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136afc3a-7508-451d-802d-27b91b13f934 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:26.447372 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136afc3a-7508-451d-802d-27b91b13f934 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:26.459753 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-136afc3a-7508-451d-802d-27b91b13f934 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:26.459753 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:26.459753 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:26.459753 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:26.459753 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:26.459753 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:26.459753 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:26.459753 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:26.459753 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:26.459753 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:26.459753 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:26.459753 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:26.459753 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 01:44:26.459753 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 01:44:26.459753 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 01:44:26.460659 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 01:44:26.460659 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:26.460659 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:26.460659 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:26.460659 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:26.460659 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:26.460659 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:26.460659 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:26.460659 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:26.460659 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:26.460659 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:26.460659 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:26.460659 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:26.460659 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:26.460659 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:26.460659 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:26.460659 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:26.461966 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 01:44:26.461966 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 01:44:26.461966 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 01:44:26.461966 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 01:44:26.461966 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 01:44:26.461966 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 01:44:26.461966 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:26.461966 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:26.461966 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 385/766] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:44:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:44:26.466725 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136afc3a-7508-451d-802d-27b91b13f934 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:26.467386 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f0e6ac-90c2-4da3-80aa-b7faf5ae51b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:26.468520 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f0e6ac-90c2-4da3-80aa-b7faf5ae51b3 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:26.469014 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f0e6ac-90c2-4da3-80aa-b7faf5ae51b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:26.469484 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f0e6ac-90c2-4da3-80aa-b7faf5ae51b3 None admin] PATH_INFO: `/v3/services/network` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:26.470346 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f0e6ac-90c2-4da3-80aa-b7faf5ae51b3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:26.472020 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f0e6ac-90c2-4da3-80aa-b7faf5ae51b3 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:26.501532 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-81f0e6ac-90c2-4da3-80aa-b7faf5ae51b3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 01:44:26.501532 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:26.501532 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 01:44:26.501532 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 01:44:26.501532 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 01:44:26.501532 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 01:44:26.501532 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 01:44:26.501532 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:44:26.501532 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 01:44:26.501532 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:26.501532 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 01:44:26.501532 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:26.501532 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:26.501532 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:26.501532 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:26.501532 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:26.501532 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:26.502555 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:26.502555 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:26.502555 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:26.502555 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:26.502555 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:26.502555 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:26.502555 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 01:44:26.502555 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 01:44:26.502555 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 01:44:26.502555 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 01:44:26.502555 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:26.502555 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:26.502555 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:26.502555 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:26.502555 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:26.502555 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:26.503480 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:26.503480 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:26.503480 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:26.503480 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:26.503480 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:26.503480 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:26.503480 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:26.503480 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:26.503480 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:26.503480 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:26.503480 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 01:44:26.503480 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:44:26.503480 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 01:44:26.503480 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:26.506467 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 382/767] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:44:26 2022] GET /identity/v3/services/network => generated 88 bytes in 395 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:26.514745 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f0e6ac-90c2-4da3-80aa-b7faf5ae51b3 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:26.553495 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:26.624848 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136afc3a-7508-451d-802d-27b91b13f934 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:26.695405 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a4c0f7-d82a-4b56-b2f9-284f74a50c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:26.696073 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a4c0f7-d82a-4b56-b2f9-284f74a50c7a None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:26.696353 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a4c0f7-d82a-4b56-b2f9-284f74a50c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:26.696625 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a4c0f7-d82a-4b56-b2f9-284f74a50c7a None admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:26.697427 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a4c0f7-d82a-4b56-b2f9-284f74a50c7a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:26.697731 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a4c0f7-d82a-4b56-b2f9-284f74a50c7a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:26.699172 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a4c0f7-d82a-4b56-b2f9-284f74a50c7a None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:26.719523 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f0e6ac-90c2-4da3-80aa-b7faf5ae51b3 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:26.728520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 386/768] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jun 23 01:44:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:26.730044 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a4c0f7-d82a-4b56-b2f9-284f74a50c7a None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:26.791487 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d31182d-cfea-43d2-a4d1-e046ad5c2de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:26.792135 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d31182d-cfea-43d2-a4d1-e046ad5c2de4 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:26.792421 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d31182d-cfea-43d2-a4d1-e046ad5c2de4 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:26.792693 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d31182d-cfea-43d2-a4d1-e046ad5c2de4 None admin] PATH_INFO: `/v3/services` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:26.793467 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d31182d-cfea-43d2-a4d1-e046ad5c2de4 None admin] RBAC: Adding query filter params (name=network) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:26.793775 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d31182d-cfea-43d2-a4d1-e046ad5c2de4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:26.795174 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d31182d-cfea-43d2-a4d1-e046ad5c2de4 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:26.806882 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 383/769] 10.222.0.26 () {56 vars in 1112 bytes} [Thu Jun 23 01:44:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:26.814363 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d31182d-cfea-43d2-a4d1-e046ad5c2de4 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:26.835179 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:26.933859 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d31182d-cfea-43d2-a4d1-e046ad5c2de4 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:26.945594 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a4c0f7-d82a-4b56-b2f9-284f74a50c7a None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:26.970964 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41932c99-3010-4e56-a79b-cecf65befd8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:26.972676 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41932c99-3010-4e56-a79b-cecf65befd8f None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:26.972956 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41932c99-3010-4e56-a79b-cecf65befd8f None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:26.973232 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41932c99-3010-4e56-a79b-cecf65befd8f None admin] PATH_INFO: `/v3/services` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:26.974050 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41932c99-3010-4e56-a79b-cecf65befd8f None admin] RBAC: Adding query filter params (type=network) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:26.974356 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41932c99-3010-4e56-a79b-cecf65befd8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:26.975858 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41932c99-3010-4e56-a79b-cecf65befd8f None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:26.982825 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 384/770] 10.222.0.26 () {56 vars in 1112 bytes} [Thu Jun 23 01:44:26 2022] GET /identity/v3/services?type=network => generated 350 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:26.988556 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41932c99-3010-4e56-a79b-cecf65befd8f None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:27.004632 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:27.031929 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84992b8-cf58-4bce-a0c7-8fdd23b1b49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:27.032838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84992b8-cf58-4bce-a0c7-8fdd23b1b49d None admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:27.033118 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84992b8-cf58-4bce-a0c7-8fdd23b1b49d None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:27.033392 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84992b8-cf58-4bce-a0c7-8fdd23b1b49d None admin] PATH_INFO: `/v3/projects/a44a5e0e3e2d4b8e9b9cec7b5b5c725c/users/ae4e83df33b64a3ebf28f54e2d94fe63/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:27.040601 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84992b8-cf58-4bce-a0c7-8fdd23b1b49d None admin] RBAC: Authorizing `identity:create_grant(project_id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c, user_id=ae4e83df33b64a3ebf28f54e2d94fe63, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:27.042070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84992b8-cf58-4bce-a0c7-8fdd23b1b49d None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:27.062585 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:27.068720 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 387/771] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jun 23 01:44:26 2022] PUT /identity/v3/projects/a44a5e0e3e2d4b8e9b9cec7b5b5c725c/users/ae4e83df33b64a3ebf28f54e2d94fe63/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 01:44:27.111881 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41932c99-3010-4e56-a79b-cecf65befd8f None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:27.132877 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad6795b-3905-40f6-bb21-c1e9465097f9 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:27.133197 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad6795b-3905-40f6-bb21-c1e9465097f9 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:27.133440 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad6795b-3905-40f6-bb21-c1e9465097f9 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:27.135163 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 388/772] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:27.140500 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37144fd-e2ab-47ff-99a3-9f5b2dffea6b None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:27.140808 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37144fd-e2ab-47ff-99a3-9f5b2dffea6b None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:27.141110 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37144fd-e2ab-47ff-99a3-9f5b2dffea6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:27.155792 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeabe57-6803-4130-abd9-c4a62340f6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:27.156550 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeabe57-6803-4130-abd9-c4a62340f6a2 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:27.156832 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeabe57-6803-4130-abd9-c4a62340f6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:27.157168 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeabe57-6803-4130-abd9-c4a62340f6a2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:27.158028 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeabe57-6803-4130-abd9-c4a62340f6a2 None admin] RBAC: Adding query filter params (interface=public, service_id=89f1d5e2348b416eb5b5d7b328f9eb82, region_id=RegionOne) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:27.158342 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeabe57-6803-4130-abd9-c4a62340f6a2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:27.159871 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeabe57-6803-4130-abd9-c4a62340f6a2 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:27.169047 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 385/773] 10.222.0.26 () {56 vars in 1251 bytes} [Thu Jun 23 01:44:26 2022] GET /identity/v3/endpoints?service_id=89f1d5e2348b416eb5b5d7b328f9eb82&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:27.174838 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeabe57-6803-4130-abd9-c4a62340f6a2 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:27.177649 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-b37144fd-e2ab-47ff-99a3-9f5b2dffea6b None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:27.178931 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-b37144fd-e2ab-47ff-99a3-9f5b2dffea6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:27.299241 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeabe57-6803-4130-abd9-c4a62340f6a2 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:27.333078 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6419f23-f7b1-45e5-a1b8-c868ae344638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:27.333747 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6419f23-f7b1-45e5-a1b8-c868ae344638 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:27.334019 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6419f23-f7b1-45e5-a1b8-c868ae344638 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:27.334290 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6419f23-f7b1-45e5-a1b8-c868ae344638 None admin] PATH_INFO: `/v3/services` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:27.334976 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6419f23-f7b1-45e5-a1b8-c868ae344638 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:27.336632 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6419f23-f7b1-45e5-a1b8-c868ae344638 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:27.337067 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 389/774] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:44:27.341104 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:27.345834 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 386/775] 10.222.0.26 () {56 vars in 1087 bytes} [Thu Jun 23 01:44:27 2022] GET /identity/v3/services => generated 772 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:44:27.361075 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37144fd-e2ab-47ff-99a3-9f5b2dffea6b None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:27.530392 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37144fd-e2ab-47ff-99a3-9f5b2dffea6b None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:27.547237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:27.569355 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87444dde-a49e-470d-b03c-2c392aba14e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:27.570176 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87444dde-a49e-470d-b03c-2c392aba14e0 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:27.570455 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87444dde-a49e-470d-b03c-2c392aba14e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:27.570729 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87444dde-a49e-470d-b03c-2c392aba14e0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:27.571424 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87444dde-a49e-470d-b03c-2c392aba14e0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:27.572819 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87444dde-a49e-470d-b03c-2c392aba14e0 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:27.585594 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-87444dde-a49e-470d-b03c-2c392aba14e0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 01:44:27.585594 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:27.585594 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 01:44:27.585594 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 01:44:27.585594 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 01:44:27.585594 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 01:44:27.585594 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 01:44:27.585594 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:44:27.585594 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 01:44:27.585594 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:27.585594 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 01:44:27.585594 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:27.585594 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:27.585594 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:27.585594 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:27.585594 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:27.586534 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:27.586534 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:27.586534 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:27.586534 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:27.586534 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:27.586534 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:27.586534 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:27.586534 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 01:44:27.586534 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 01:44:27.586534 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 01:44:27.586534 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 01:44:27.586534 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:27.586534 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:27.586534 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:27.586534 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:27.586534 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:27.587445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:27.587445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:27.587445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:27.587445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:27.587445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:27.587445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:27.587445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:27.587445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:27.587445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:27.587445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:27.587445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:27.587445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 01:44:27.587445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:44:27.587445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 01:44:27.587445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:27.587445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 390/776] 10.222.0.26 () {56 vars in 1129 bytes} [Thu Jun 23 01:44:27 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:27.594619 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6419f23-f7b1-45e5-a1b8-c868ae344638 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:27.723638 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6419f23-f7b1-45e5-a1b8-c868ae344638 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:27.772609 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d357577d-6f4c-46cc-88b5-8ed71dcb700f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:27.773284 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d357577d-6f4c-46cc-88b5-8ed71dcb700f None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:27.773569 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d357577d-6f4c-46cc-88b5-8ed71dcb700f None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:27.773845 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d357577d-6f4c-46cc-88b5-8ed71dcb700f None admin] PATH_INFO: `/v3/services` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:27.774559 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d357577d-6f4c-46cc-88b5-8ed71dcb700f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:27.774867 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d357577d-6f4c-46cc-88b5-8ed71dcb700f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:27.776274 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d357577d-6f4c-46cc-88b5-8ed71dcb700f None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:27.780973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 387/777] 10.222.0.26 () {56 vars in 1124 bytes} [Thu Jun 23 01:44:27 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:27.786776 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87444dde-a49e-470d-b03c-2c392aba14e0 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:27.954853 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87444dde-a49e-470d-b03c-2c392aba14e0 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:27.999825 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47dc7b1-61fd-41a0-ad10-4c164ab7728e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:28.000516 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47dc7b1-61fd-41a0-ad10-4c164ab7728e None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:28.000893 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47dc7b1-61fd-41a0-ad10-4c164ab7728e None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:28.001181 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47dc7b1-61fd-41a0-ad10-4c164ab7728e None admin] PATH_INFO: `/v3/services` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:28.001977 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47dc7b1-61fd-41a0-ad10-4c164ab7728e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:28.002286 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47dc7b1-61fd-41a0-ad10-4c164ab7728e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:28.003993 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47dc7b1-61fd-41a0-ad10-4c164ab7728e None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:28.015933 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 391/778] 10.222.0.26 () {56 vars in 1124 bytes} [Thu Jun 23 01:44:27 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:28.022118 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d357577d-6f4c-46cc-88b5-8ed71dcb700f None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:28.078811 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ec113-e373-431a-9349-40210837786d None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:28.079321 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ec113-e373-431a-9349-40210837786d None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:28.079611 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ec113-e373-431a-9349-40210837786d None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:28.082567 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 392/779] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:28.086659 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1538a0-9a9b-4581-8893-d735c1c23b03 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:28.086956 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1538a0-9a9b-4581-8893-d735c1c23b03 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:28.087239 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1538a0-9a9b-4581-8893-d735c1c23b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:28.110050 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:28.115171 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-4a1538a0-9a9b-4581-8893-d735c1c23b03 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:28.116436 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-4a1538a0-9a9b-4581-8893-d735c1c23b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:28.160571 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:28.173969 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d357577d-6f4c-46cc-88b5-8ed71dcb700f None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:28.214951 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa11cf46-458a-4d09-8dbf-44ba6dfbd0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:28.215597 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11cf46-458a-4d09-8dbf-44ba6dfbd0cf None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:28.215879 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11cf46-458a-4d09-8dbf-44ba6dfbd0cf None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:28.216151 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11cf46-458a-4d09-8dbf-44ba6dfbd0cf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:28.216953 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa11cf46-458a-4d09-8dbf-44ba6dfbd0cf None admin] RBAC: Adding query filter params (interface=public, service_id=16ddfcfe6649412393ff1930982c6e6b, region_id=RegionOne) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:28.217263 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa11cf46-458a-4d09-8dbf-44ba6dfbd0cf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:28.218670 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa11cf46-458a-4d09-8dbf-44ba6dfbd0cf None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:28.228808 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 388/780] 10.222.0.26 () {56 vars in 1251 bytes} [Thu Jun 23 01:44:28 2022] GET /identity/v3/endpoints?service_id=16ddfcfe6649412393ff1930982c6e6b&interface=public®ion_id=RegionOne => generated 194 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:28.234391 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa11cf46-458a-4d09-8dbf-44ba6dfbd0cf None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:28.249401 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 393/781] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 01:44:28.305644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1538a0-9a9b-4581-8893-d735c1c23b03 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:28.434041 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa11cf46-458a-4d09-8dbf-44ba6dfbd0cf None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:28.484573 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1538a0-9a9b-4581-8893-d735c1c23b03 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:28.508777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:28.525178 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c4d05-1be9-4b82-adf9-b761494408fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:28.525871 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c4d05-1be9-4b82-adf9-b761494408fb None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:28.526153 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c4d05-1be9-4b82-adf9-b761494408fb None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:28.526530 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c4d05-1be9-4b82-adf9-b761494408fb None admin] PATH_INFO: `/v3/services` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:28.527260 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9c4d05-1be9-4b82-adf9-b761494408fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:28.528679 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9c4d05-1be9-4b82-adf9-b761494408fb None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:28.537922 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309a5969-e003-4510-9d11-cf811c954037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:28.538726 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309a5969-e003-4510-9d11-cf811c954037 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:28.539009 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309a5969-e003-4510-9d11-cf811c954037 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:28.539290 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 389/782] 10.222.0.26 () {56 vars in 1087 bytes} [Thu Jun 23 01:44:28 2022] GET /identity/v3/services => generated 772 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:28.547551 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309a5969-e003-4510-9d11-cf811c954037 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:28.572575 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309a5969-e003-4510-9d11-cf811c954037 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:44:28.580925 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-309a5969-e003-4510-9d11-cf811c954037 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:44:28.580925 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:28.580925 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:28.580925 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:28.580925 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:28.580925 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:28.580925 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:28.580925 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:28.580925 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:28.580925 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:28.580925 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:28.580925 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:28.580925 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:44:28.580925 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:44:28.580925 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:44:28.580925 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:44:28.581959 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:44:28.581959 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:44:28.581959 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:44:28.581959 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:44:28.581959 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:28.581959 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:28.581959 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:28.581959 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:28.581959 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:28.581959 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:28.581959 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:28.581959 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:28.581959 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:28.581959 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:28.581959 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:28.581959 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:28.582783 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:28.582783 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:28.582783 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:28.582783 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:28.582783 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:44:28.582783 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:44:28.582783 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:44:28.582783 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:44:28.582783 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:44:28.582783 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:44:28.582783 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:44:28.582783 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:28.582783 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 394/783] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jun 23 01:44:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:44:28.588873 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c4d05-1be9-4b82-adf9-b761494408fb None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:28.759895 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c4d05-1be9-4b82-adf9-b761494408fb None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:28.764153 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e7708a-0440-463d-8202-53ba653e679c None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:28.764442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e7708a-0440-463d-8202-53ba653e679c None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:28.764717 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e7708a-0440-463d-8202-53ba653e679c None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:28.766420 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 395/784] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:28.771634 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb5c8e-fff1-4016-bd4d-08eaf8b7922d None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:28.771919 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb5c8e-fff1-4016-bd4d-08eaf8b7922d None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:28.772197 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb5c8e-fff1-4016-bd4d-08eaf8b7922d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:28.801261 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-70fb5c8e-fff1-4016-bd4d-08eaf8b7922d None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:28.802484 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-70fb5c8e-fff1-4016-bd4d-08eaf8b7922d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:28.824897 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c38271-33a0-4378-835f-d8f1d4819fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:28.825578 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c38271-33a0-4378-835f-d8f1d4819fbf None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:28.825858 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c38271-33a0-4378-835f-d8f1d4819fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:28.826132 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c38271-33a0-4378-835f-d8f1d4819fbf None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:28.826893 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c38271-33a0-4378-835f-d8f1d4819fbf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:28.827203 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c38271-33a0-4378-835f-d8f1d4819fbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:28.828641 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c38271-33a0-4378-835f-d8f1d4819fbf None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:28.836271 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:28.838723 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 390/785] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:44:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:28.848231 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c38271-33a0-4378-835f-d8f1d4819fbf None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:28.886257 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 396/786] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:44:28.910791 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fb5c8e-fff1-4016-bd4d-08eaf8b7922d None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:28.913676 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:28.972298 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c38271-33a0-4378-835f-d8f1d4819fbf None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:29.044809 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ad2750-d546-46ca-b5e6-7d60e4c95c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:29.045591 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad2750-d546-46ca-b5e6-7d60e4c95c20 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:29.045876 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad2750-d546-46ca-b5e6-7d60e4c95c20 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:29.046154 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad2750-d546-46ca-b5e6-7d60e4c95c20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:29.061487 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ad2750-d546-46ca-b5e6-7d60e4c95c20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:29.062970 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ad2750-d546-46ca-b5e6-7d60e4c95c20 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:29.082627 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-b5ad2750-d546-46ca-b5e6-7d60e4c95c20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:29.082627 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:29.082627 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:29.082627 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:29.082627 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:29.082627 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:29.082627 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:29.082627 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:29.082627 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:29.082627 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:29.082627 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:29.082627 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:29.082627 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:29.082627 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:29.082627 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:29.083571 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:29.083571 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:29.083571 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:29.083571 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:29.083571 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:29.083571 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:29.083571 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:29.083571 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:29.083571 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:29.083571 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:29.083571 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:29.083571 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:29.083571 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:29.083571 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:29.083571 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:29.083571 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:29.083571 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:29.084441 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:29.084441 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:29.084441 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:29.084441 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:29.089017 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 391/787] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:29.090256 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ad2750-d546-46ca-b5e6-7d60e4c95c20 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:29.125236 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fb5c8e-fff1-4016-bd4d-08eaf8b7922d None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:29.196112 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ad2750-d546-46ca-b5e6-7d60e4c95c20 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:29.208009 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:29.222204 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35376bd-63d1-4119-8f93-03445d71f16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:29.223016 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35376bd-63d1-4119-8f93-03445d71f16a None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:29.223345 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35376bd-63d1-4119-8f93-03445d71f16a None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:29.223624 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35376bd-63d1-4119-8f93-03445d71f16a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:29.235028 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35376bd-63d1-4119-8f93-03445d71f16a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:44:29.243200 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-a35376bd-63d1-4119-8f93-03445d71f16a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 01:44:29.243200 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:29.243200 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:29.243200 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:29.243200 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:29.243200 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:29.243200 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:29.243200 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:29.243200 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:29.243200 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:29.243200 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:29.243200 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:29.243200 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:44:29.243200 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:44:29.243200 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:44:29.243200 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:44:29.244136 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:44:29.244136 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:44:29.244136 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:44:29.244136 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:44:29.244136 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:29.244136 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:29.244136 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:29.244136 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:29.244136 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:29.244136 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:29.244136 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:29.244136 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:29.244136 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:29.244136 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:29.244136 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:29.244136 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:29.244938 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:29.244938 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:29.244938 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:29.244938 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:29.244938 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:44:29.244938 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:44:29.244938 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:44:29.244938 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:44:29.244938 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:44:29.244938 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:44:29.244938 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 01:44:29.244938 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:29.244938 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 397/788] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jun 23 01:44:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:29.253050 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35376bd-63d1-4119-8f93-03445d71f16a None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:29.265264 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c92f5c-9746-42e1-835c-7d82a2c0242b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:29.265978 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c92f5c-9746-42e1-835c-7d82a2c0242b None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:29.266265 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c92f5c-9746-42e1-835c-7d82a2c0242b None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:29.266546 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c92f5c-9746-42e1-835c-7d82a2c0242b None admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:29.270747 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c92f5c-9746-42e1-835c-7d82a2c0242b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:29.271067 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c92f5c-9746-42e1-835c-7d82a2c0242b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:29.272554 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c92f5c-9746-42e1-835c-7d82a2c0242b None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:29.311588 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 392/789] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:29.314542 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:29.318975 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c92f5c-9746-42e1-835c-7d82a2c0242b None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:29.468023 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35376bd-63d1-4119-8f93-03445d71f16a None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:29.515755 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c92f5c-9746-42e1-835c-7d82a2c0242b None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:29.539199 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce23b92e-4dd6-4a68-8523-9b9f46596543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:29.539901 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23b92e-4dd6-4a68-8523-9b9f46596543 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:29.540183 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23b92e-4dd6-4a68-8523-9b9f46596543 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:29.540459 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23b92e-4dd6-4a68-8523-9b9f46596543 None admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:29.541230 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce23b92e-4dd6-4a68-8523-9b9f46596543 None admin] RBAC: Adding query filter params (name=service) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:29.541537 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce23b92e-4dd6-4a68-8523-9b9f46596543 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:29.542934 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce23b92e-4dd6-4a68-8523-9b9f46596543 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:29.560914 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 398/790] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Jun 23 01:44:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:44:29.562309 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76809d2-0775-4fa3-a87f-09aba6b4cf25 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:29.562540 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76809d2-0775-4fa3-a87f-09aba6b4cf25 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:29.562817 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76809d2-0775-4fa3-a87f-09aba6b4cf25 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:29.567297 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 399/791] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:29.569087 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76809d2-0775-4fa3-a87f-09aba6b4cf25 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:29.573079 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff64f86-eccf-4cb0-8cff-3ad6a8a09a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:29.573883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff64f86-eccf-4cb0-8cff-3ad6a8a09a38 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:29.574165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff64f86-eccf-4cb0-8cff-3ad6a8a09a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:29.574440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff64f86-eccf-4cb0-8cff-3ad6a8a09a38 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:29.581996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff64f86-eccf-4cb0-8cff-3ad6a8a09a38 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:29.583505 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff64f86-eccf-4cb0-8cff-3ad6a8a09a38 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:29.598426 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-2ff64f86-eccf-4cb0-8cff-3ad6a8a09a38 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 01:44:29.598426 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:29.598426 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:29.598426 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:29.598426 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:29.598426 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:29.598426 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:29.598426 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:29.598426 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:29.598426 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:29.598426 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:29.598426 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:29.598426 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 01:44:29.598426 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 01:44:29.598426 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 01:44:29.598426 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 01:44:29.599467 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:29.599467 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:29.599467 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 01:44:29.599467 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:29.599467 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 01:44:29.599467 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:29.599467 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:29.599467 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:29.599467 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:29.599467 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:29.599467 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:29.599467 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:29.599467 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:29.599467 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:29.599467 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:29.599467 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:29.599467 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:29.599467 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:29.600438 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:29.600438 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:29.600438 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 01:44:29.600438 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 01:44:29.600438 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 01:44:29.600438 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 01:44:29.600438 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 01:44:29.600438 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 01:44:29.600438 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 01:44:29.600438 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:29.600438 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 393/792] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jun 23 01:44:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:29.601551 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f164da86-883a-42ba-8a92-c853abf9bc4d None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:29.601551 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f164da86-883a-42ba-8a92-c853abf9bc4d None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:29.601782 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f164da86-883a-42ba-8a92-c853abf9bc4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:29.623879 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-f164da86-883a-42ba-8a92-c853abf9bc4d None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:29.625070 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-f164da86-883a-42ba-8a92-c853abf9bc4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:29.646756 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:29.688744 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:29.706590 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76809d2-0775-4fa3-a87f-09aba6b4cf25 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:29.714887 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 394/793] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:44:29.716088 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f164da86-883a-42ba-8a92-c853abf9bc4d None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:29.756858 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d1e41-65bf-4a84-aa8c-218bf6485041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:29.757660 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d1e41-65bf-4a84-aa8c-218bf6485041 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:29.757947 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d1e41-65bf-4a84-aa8c-218bf6485041 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:29.758224 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d1e41-65bf-4a84-aa8c-218bf6485041 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:29.769423 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d1e41-65bf-4a84-aa8c-218bf6485041 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:29.770899 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d1e41-65bf-4a84-aa8c-218bf6485041 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:29.788034 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-9a7d1e41-65bf-4a84-aa8c-218bf6485041 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:29.788034 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:29.788034 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:29.788034 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:29.788034 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:29.788034 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:29.788034 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:29.788034 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:29.788034 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:29.788034 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:29.788034 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:29.788034 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:29.788034 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:29.788034 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:29.788034 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:29.788900 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:29.788900 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:29.788900 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:29.788900 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:29.788900 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:29.788900 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:29.788900 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:29.788900 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:29.788900 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:29.788900 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:29.788900 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:29.788900 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:29.788900 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:29.788900 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:29.788900 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:29.788900 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:29.788900 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:29.790455 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:29.790455 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:29.790455 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:29.790455 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:29.790455 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 400/794] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:44:29.790455 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d1e41-65bf-4a84-aa8c-218bf6485041 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:29.840585 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f164da86-883a-42ba-8a92-c853abf9bc4d None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:29.879123 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1bff71-5d50-4522-b6a0-c02fd8d0914d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:29.880052 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1bff71-5d50-4522-b6a0-c02fd8d0914d None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:29.880794 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1bff71-5d50-4522-b6a0-c02fd8d0914d None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:29.881570 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1bff71-5d50-4522-b6a0-c02fd8d0914d None admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:29.882559 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1bff71-5d50-4522-b6a0-c02fd8d0914d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:29.883006 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1bff71-5d50-4522-b6a0-c02fd8d0914d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:29.885146 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1bff71-5d50-4522-b6a0-c02fd8d0914d None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:29.896071 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d1e41-65bf-4a84-aa8c-218bf6485041 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:29.909764 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 395/795] 10.222.0.26 () {56 vars in 1135 bytes} [Thu Jun 23 01:44:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:29.911316 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c439817b-6d8c-493e-841e-c69ec253853b None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:29.911627 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c439817b-6d8c-493e-841e-c69ec253853b None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:29.911938 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c439817b-6d8c-493e-841e-c69ec253853b None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:29.913273 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 396/796] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:29.917467 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c439817b-6d8c-493e-841e-c69ec253853b None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:29.931333 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:29.936617 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8683d23c-645d-4f1c-9941-259bc607f39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:29.937448 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8683d23c-645d-4f1c-9941-259bc607f39c None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:29.937735 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8683d23c-645d-4f1c-9941-259bc607f39c None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:29.937996 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8683d23c-645d-4f1c-9941-259bc607f39c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:29.940776 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:29.949643 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8683d23c-645d-4f1c-9941-259bc607f39c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:44:29.957472 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-8683d23c-645d-4f1c-9941-259bc607f39c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:44:29.957472 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:29.957472 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:29.957472 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:29.957472 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:29.957472 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:29.957472 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:29.957472 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:29.957472 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:29.957472 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:29.957472 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:29.957472 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:29.957472 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:44:29.957472 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:44:29.957472 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:44:29.957472 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:44:29.958397 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:44:29.958397 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:44:29.958397 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:44:29.958397 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:44:29.958397 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:29.958397 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:29.958397 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:29.958397 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:29.958397 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:29.958397 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:29.958397 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:29.958397 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:29.958397 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:29.958397 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:29.958397 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:29.958397 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:29.959237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:29.959237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:29.959237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:29.959237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:29.959237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:44:29.959237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:44:29.959237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:44:29.959237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:44:29.959237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:44:29.959237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:44:29.959237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:44:29.959237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:29.963559 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 401/797] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jun 23 01:44:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:29.964240 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8683d23c-645d-4f1c-9941-259bc607f39c None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:30.025015 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c439817b-6d8c-493e-841e-c69ec253853b None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:30.061341 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b985439-7285-43fc-92b3-968f4332a0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:30.062009 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b985439-7285-43fc-92b3-968f4332a0e6 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:30.062293 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b985439-7285-43fc-92b3-968f4332a0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:30.062570 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b985439-7285-43fc-92b3-968f4332a0e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:30.063320 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b985439-7285-43fc-92b3-968f4332a0e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:30.063632 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b985439-7285-43fc-92b3-968f4332a0e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:30.065003 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b985439-7285-43fc-92b3-968f4332a0e6 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:30.076378 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 397/798] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:30.077604 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0a6a3-bca0-4bb9-b102-20bce2337816 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:30.077891 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0a6a3-bca0-4bb9-b102-20bce2337816 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:30.078164 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0a6a3-bca0-4bb9-b102-20bce2337816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:30.084730 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8683d23c-645d-4f1c-9941-259bc607f39c None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:30.096495 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-2db0a6a3-bca0-4bb9-b102-20bce2337816 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:30.097623 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-2db0a6a3-bca0-4bb9-b102-20bce2337816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:30.121763 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b945aaef-e36a-46fd-a2a5-14f17046471a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:30.122551 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b945aaef-e36a-46fd-a2a5-14f17046471a None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:30.122833 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b945aaef-e36a-46fd-a2a5-14f17046471a None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:30.123093 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b945aaef-e36a-46fd-a2a5-14f17046471a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:30.134173 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b945aaef-e36a-46fd-a2a5-14f17046471a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:30.135654 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b945aaef-e36a-46fd-a2a5-14f17046471a None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:30.152642 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-b945aaef-e36a-46fd-a2a5-14f17046471a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:30.152642 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:30.152642 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:30.152642 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:30.152642 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:30.152642 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:30.152642 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:30.152642 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:30.152642 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:30.152642 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:30.152642 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:30.152642 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:30.152642 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:30.152642 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:30.152642 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:30.153574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:30.153574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:30.153574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:30.153574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:30.153574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:30.153574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:30.153574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:30.153574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:30.153574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:30.153574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:30.153574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:30.153574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:30.153574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:30.153574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:30.153574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:30.153574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:30.153574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:30.154502 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:30.154502 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:30.154502 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:30.154502 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:30.154502 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 402/799] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:30.155271 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b945aaef-e36a-46fd-a2a5-14f17046471a None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:30.181625 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 398/800] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:44:30.181625 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db0a6a3-bca0-4bb9-b102-20bce2337816 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:30.186022 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:30.207392 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:30.261884 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b945aaef-e36a-46fd-a2a5-14f17046471a None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:30.293777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db0a6a3-bca0-4bb9-b102-20bce2337816 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:30.307169 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e406a59b-2bdc-4285-a9f5-89eb225d9aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:30.308114 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e406a59b-2bdc-4285-a9f5-89eb225d9aab None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:30.308856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e406a59b-2bdc-4285-a9f5-89eb225d9aab None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:30.309141 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e406a59b-2bdc-4285-a9f5-89eb225d9aab None admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:30.310176 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e406a59b-2bdc-4285-a9f5-89eb225d9aab None admin] RBAC: Adding query filter params (name=admin) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:30.310488 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e406a59b-2bdc-4285-a9f5-89eb225d9aab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:30.312537 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e406a59b-2bdc-4285-a9f5-89eb225d9aab None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:30.328816 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 403/801] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:44:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:30.328816 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e406a59b-2bdc-4285-a9f5-89eb225d9aab None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:30.334504 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9ab8e4-7119-4369-a250-89898831b17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:30.335664 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ab8e4-7119-4369-a250-89898831b17d None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:30.336115 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ab8e4-7119-4369-a250-89898831b17d None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:30.336402 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ab8e4-7119-4369-a250-89898831b17d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:30.345043 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9ab8e4-7119-4369-a250-89898831b17d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:30.346584 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9ab8e4-7119-4369-a250-89898831b17d None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:30.364921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-6a9ab8e4-7119-4369-a250-89898831b17d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 01:44:30.364921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:30.364921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:30.364921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:30.364921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:30.364921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:30.364921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:30.364921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:30.364921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:30.364921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:30.364921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:30.364921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:30.364921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 01:44:30.364921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 01:44:30.364921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 01:44:30.364921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 01:44:30.366128 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:30.366128 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:30.366128 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 01:44:30.366128 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:30.366128 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 01:44:30.366128 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:30.366128 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:30.366128 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:30.366128 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:30.366128 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:30.366128 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:30.366128 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:30.366128 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:30.366128 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:30.366128 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:30.366128 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:30.366128 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:30.366128 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:30.367274 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:30.367274 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:30.367274 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 01:44:30.367274 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 01:44:30.367274 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 01:44:30.367274 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 01:44:30.367274 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 01:44:30.367274 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 01:44:30.367274 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 01:44:30.367274 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:30.367274 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 399/802] 10.222.0.26 () {56 vars in 1117 bytes} [Thu Jun 23 01:44:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:30.368433 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9ab8e4-7119-4369-a250-89898831b17d None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:30.437644 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:30.446075 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e406a59b-2bdc-4285-a9f5-89eb225d9aab None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:30.458173 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:30.479231 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9ab8e4-7119-4369-a250-89898831b17d None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:30.482491 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd08e64-112f-41b2-86d9-7203dd2ae587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:30.483150 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd08e64-112f-41b2-86d9-7203dd2ae587 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:30.483473 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd08e64-112f-41b2-86d9-7203dd2ae587 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:30.483745 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd08e64-112f-41b2-86d9-7203dd2ae587 None admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:30.484465 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd08e64-112f-41b2-86d9-7203dd2ae587 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:30.484766 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd08e64-112f-41b2-86d9-7203dd2ae587 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:30.486150 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd08e64-112f-41b2-86d9-7203dd2ae587 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:30.502223 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 404/803] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:30.502223 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd08e64-112f-41b2-86d9-7203dd2ae587 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:30.517324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db51b843-0ccb-4fc0-92b5-e91e6cd72f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:30.518239 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db51b843-0ccb-4fc0-92b5-e91e6cd72f7c None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:30.518558 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db51b843-0ccb-4fc0-92b5-e91e6cd72f7c None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:30.518836 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db51b843-0ccb-4fc0-92b5-e91e6cd72f7c None admin] PATH_INFO: `/v3/services/network` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:30.519524 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db51b843-0ccb-4fc0-92b5-e91e6cd72f7c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:30.521147 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db51b843-0ccb-4fc0-92b5-e91e6cd72f7c None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:30.533762 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-db51b843-0ccb-4fc0-92b5-e91e6cd72f7c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 01:44:30.533762 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:30.533762 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 01:44:30.533762 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 01:44:30.533762 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 01:44:30.533762 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 01:44:30.533762 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 01:44:30.533762 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:44:30.533762 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 01:44:30.533762 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:30.533762 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 01:44:30.533762 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:30.533762 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:30.533762 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:30.533762 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:30.533762 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:30.533762 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:30.534820 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:30.534820 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:30.534820 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:30.534820 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:30.534820 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:30.534820 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:30.534820 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 01:44:30.534820 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 01:44:30.534820 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 01:44:30.534820 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 01:44:30.534820 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:30.534820 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:30.534820 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:30.534820 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:30.534820 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:30.534820 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:30.535681 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:30.535681 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:30.535681 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:30.535681 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:30.535681 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:30.535681 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:30.535681 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:30.535681 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:30.535681 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:30.535681 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:30.535681 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 01:44:30.535681 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:44:30.535681 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 01:44:30.535681 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:30.540024 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 400/804] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:44:30 2022] GET /identity/v3/services/network => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:30.540898 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db51b843-0ccb-4fc0-92b5-e91e6cd72f7c None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:30.604347 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd08e64-112f-41b2-86d9-7203dd2ae587 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:30.637991 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc33f89-90d6-4494-a960-b47d48152dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:30.638774 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc33f89-90d6-4494-a960-b47d48152dcf None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:30.639053 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc33f89-90d6-4494-a960-b47d48152dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:30.639353 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc33f89-90d6-4494-a960-b47d48152dcf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:30.648058 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db51b843-0ccb-4fc0-92b5-e91e6cd72f7c None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:30.649259 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc33f89-90d6-4494-a960-b47d48152dcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:30.650733 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc33f89-90d6-4494-a960-b47d48152dcf None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:30.668247 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-1bc33f89-90d6-4494-a960-b47d48152dcf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:30.668247 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:30.668247 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:30.668247 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:30.668247 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:30.668247 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:30.668247 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:30.668247 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:30.668247 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:30.668247 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:30.668247 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:30.668247 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:30.668247 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:30.668247 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:30.668247 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:30.669136 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:30.669136 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:30.669136 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:30.669136 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:30.669136 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:30.669136 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:30.669136 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:30.669136 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:30.669136 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:30.669136 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:30.669136 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:30.669136 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:30.669136 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:30.669136 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:30.669136 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:30.669136 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:30.669136 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:30.670024 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:30.670024 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:30.670024 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:30.670024 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:30.670024 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 405/805] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:30.670303 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc33f89-90d6-4494-a960-b47d48152dcf None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:30.682462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:30.692436 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dca0c86-ff98-42b9-9e3d-ce83782a6612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:30.693435 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca0c86-ff98-42b9-9e3d-ce83782a6612 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:30.693863 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca0c86-ff98-42b9-9e3d-ce83782a6612 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:30.694285 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca0c86-ff98-42b9-9e3d-ce83782a6612 None admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:30.695309 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dca0c86-ff98-42b9-9e3d-ce83782a6612 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:30.695793 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dca0c86-ff98-42b9-9e3d-ce83782a6612 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:30.697432 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dca0c86-ff98-42b9-9e3d-ce83782a6612 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:30.698835 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:30.722726 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 401/806] 10.222.0.26 () {56 vars in 1149 bytes} [Thu Jun 23 01:44:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:30.723469 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dca0c86-ff98-42b9-9e3d-ce83782a6612 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:30.778767 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc33f89-90d6-4494-a960-b47d48152dcf None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:30.817169 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eea5e4-acd1-474a-8285-943c00a980ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:30.817974 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eea5e4-acd1-474a-8285-943c00a980ad None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:30.818255 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eea5e4-acd1-474a-8285-943c00a980ad None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:30.818531 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eea5e4-acd1-474a-8285-943c00a980ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:30.826408 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eea5e4-acd1-474a-8285-943c00a980ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:30.827896 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eea5e4-acd1-474a-8285-943c00a980ad None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:30.828716 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dca0c86-ff98-42b9-9e3d-ce83782a6612 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:30.841574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-32eea5e4-acd1-474a-8285-943c00a980ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:30.841574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:30.841574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:30.841574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:30.841574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:30.841574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:30.841574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:30.841574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:30.841574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:30.841574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:30.841574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:30.841574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:30.841574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 01:44:30.841574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 01:44:30.841574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 01:44:30.842470 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 01:44:30.842470 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:30.842470 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:30.842470 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:30.842470 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:30.842470 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:30.842470 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:30.842470 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:30.842470 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:30.842470 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:30.842470 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:30.842470 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:30.842470 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:30.842470 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:30.842470 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:30.842470 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:30.842470 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:30.843891 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 01:44:30.843891 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 01:44:30.843891 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 01:44:30.843891 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 01:44:30.843891 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 01:44:30.843891 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 01:44:30.843891 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:30.843891 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:30.843891 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 406/807] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:44:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:30.843891 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eea5e4-acd1-474a-8285-943c00a980ad None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:30.873666 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3b89f5-1f58-49ba-be5a-8d47796cf93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:30.874426 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b89f5-1f58-49ba-be5a-8d47796cf93e None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:30.874811 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b89f5-1f58-49ba-be5a-8d47796cf93e None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:30.875233 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b89f5-1f58-49ba-be5a-8d47796cf93e None admin] PATH_INFO: `/v3/services` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:30.875984 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3b89f5-1f58-49ba-be5a-8d47796cf93e None admin] RBAC: Adding query filter params (name=network) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:30.876362 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3b89f5-1f58-49ba-be5a-8d47796cf93e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:30.877804 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3b89f5-1f58-49ba-be5a-8d47796cf93e None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:30.886892 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 402/808] 10.222.0.26 () {56 vars in 1112 bytes} [Thu Jun 23 01:44:30 2022] GET /identity/v3/services?name=network => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:30.886892 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3b89f5-1f58-49ba-be5a-8d47796cf93e None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:30.933665 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:30.933863 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:30.951467 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eea5e4-acd1-474a-8285-943c00a980ad None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:30.982857 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3b89f5-1f58-49ba-be5a-8d47796cf93e None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:30.984721 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d861d204-764d-413f-9915-bc113b06c738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:30.985442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861d204-764d-413f-9915-bc113b06c738 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:30.985726 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861d204-764d-413f-9915-bc113b06c738 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:30.986002 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861d204-764d-413f-9915-bc113b06c738 None admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:30.986729 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d861d204-764d-413f-9915-bc113b06c738 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:30.987114 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d861d204-764d-413f-9915-bc113b06c738 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:30.988537 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d861d204-764d-413f-9915-bc113b06c738 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:31.000781 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 407/809] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:31.001574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d861d204-764d-413f-9915-bc113b06c738 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:31.020971 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c7967-96a6-422c-a429-3033351ba9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:31.021773 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c7967-96a6-422c-a429-3033351ba9af None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:31.022059 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c7967-96a6-422c-a429-3033351ba9af None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:31.022341 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c7967-96a6-422c-a429-3033351ba9af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:31.032662 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5c7967-96a6-422c-a429-3033351ba9af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:31.034115 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5c7967-96a6-422c-a429-3033351ba9af None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:31.051652 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-cf5c7967-96a6-422c-a429-3033351ba9af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:31.051652 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:31.051652 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:31.051652 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:31.051652 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:31.051652 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:31.051652 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:31.051652 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:31.051652 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:31.051652 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:31.051652 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:31.051652 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:31.051652 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:31.051652 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:31.051652 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:31.052740 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:31.052740 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:31.052740 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:31.052740 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:31.052740 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:31.052740 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:31.052740 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:31.052740 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:31.052740 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:31.052740 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:31.052740 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:31.052740 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:31.052740 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:31.052740 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:31.052740 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:31.052740 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:31.052740 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:31.053816 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:31.053816 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:31.053816 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:31.053816 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:31.053816 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 403/810] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:31.053816 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c7967-96a6-422c-a429-3033351ba9af None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:31.102037 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d861d204-764d-413f-9915-bc113b06c738 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:31.135082 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2957d8-756b-411a-a3c9-9f57a757d6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:31.135755 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2957d8-756b-411a-a3c9-9f57a757d6cc None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:31.135972 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2957d8-756b-411a-a3c9-9f57a757d6cc None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:31.136240 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2957d8-756b-411a-a3c9-9f57a757d6cc None admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:31.137023 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2957d8-756b-411a-a3c9-9f57a757d6cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:31.137328 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2957d8-756b-411a-a3c9-9f57a757d6cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:31.138775 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2957d8-756b-411a-a3c9-9f57a757d6cc None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:31.150220 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 408/811] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jun 23 01:44:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:31.151438 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6c812-ce08-44b1-8248-3e61d722cb2c None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:31.151725 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6c812-ce08-44b1-8248-3e61d722cb2c None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:31.152005 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6c812-ce08-44b1-8248-3e61d722cb2c None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:31.153221 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 409/812] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:31.153911 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6c812-ce08-44b1-8248-3e61d722cb2c None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:31.156489 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:31.159705 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c7967-96a6-422c-a429-3033351ba9af None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:31.161424 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:31.196276 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d979093-4a45-4b53-9395-4f1636372161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:31.196938 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d979093-4a45-4b53-9395-4f1636372161 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:31.197212 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d979093-4a45-4b53-9395-4f1636372161 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:31.197481 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d979093-4a45-4b53-9395-4f1636372161 None admin] PATH_INFO: `/v3/services` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:31.198189 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d979093-4a45-4b53-9395-4f1636372161 None admin] RBAC: Adding query filter params (type=network) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:31.198511 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d979093-4a45-4b53-9395-4f1636372161 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:31.199905 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d979093-4a45-4b53-9395-4f1636372161 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:31.204925 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 404/813] 10.222.0.26 () {56 vars in 1112 bytes} [Thu Jun 23 01:44:31 2022] GET /identity/v3/services?type=network => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:31.205573 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d979093-4a45-4b53-9395-4f1636372161 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:31.255646 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6c812-ce08-44b1-8248-3e61d722cb2c None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:31.288891 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab0fc0-a8d1-417a-8545-1f410977c80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:31.292437 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab0fc0-a8d1-417a-8545-1f410977c80c None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:31.292437 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab0fc0-a8d1-417a-8545-1f410977c80c None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:31.292437 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ab0fc0-a8d1-417a-8545-1f410977c80c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:31.299856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ab0fc0-a8d1-417a-8545-1f410977c80c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:31.301342 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ab0fc0-a8d1-417a-8545-1f410977c80c None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:31.308484 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d979093-4a45-4b53-9395-4f1636372161 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:31.315602 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-48ab0fc0-a8d1-417a-8545-1f410977c80c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 01:44:31.315602 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:31.315602 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:31.315602 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:31.315602 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:31.315602 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:31.315602 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:31.315602 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:31.315602 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:31.315602 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:31.315602 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:31.315602 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:31.315602 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 01:44:31.315602 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 01:44:31.315602 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 01:44:31.315602 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 01:44:31.316693 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:31.316693 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:31.316693 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 01:44:31.316693 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:31.316693 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 01:44:31.316693 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:31.316693 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:31.316693 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:31.316693 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:31.316693 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:31.316693 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:31.316693 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:31.316693 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:31.316693 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:31.316693 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:31.316693 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:31.316693 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:31.316693 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:31.317823 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:31.317823 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:31.317823 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 01:44:31.317823 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 01:44:31.317823 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 01:44:31.317823 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 01:44:31.317823 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 01:44:31.317823 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 01:44:31.317823 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 01:44:31.317823 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:31.319499 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 410/814] 10.222.0.26 () {56 vars in 1093 bytes} [Thu Jun 23 01:44:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:31.320139 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab0fc0-a8d1-417a-8545-1f410977c80c None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:31.343581 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d936cf6c-32bc-46af-bc42-59805a0fcb50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:31.344452 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936cf6c-32bc-46af-bc42-59805a0fcb50 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:31.344735 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936cf6c-32bc-46af-bc42-59805a0fcb50 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:31.345008 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936cf6c-32bc-46af-bc42-59805a0fcb50 None admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:31.345736 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d936cf6c-32bc-46af-bc42-59805a0fcb50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:31.346041 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d936cf6c-32bc-46af-bc42-59805a0fcb50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:31.347534 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d936cf6c-32bc-46af-bc42-59805a0fcb50 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:31.358616 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 405/815] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:31.359818 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb3a69-7f7e-4824-bbce-a998f9a0025c None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:31.360095 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb3a69-7f7e-4824-bbce-a998f9a0025c None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:31.360379 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb3a69-7f7e-4824-bbce-a998f9a0025c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:31.377388 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-a4cb3a69-7f7e-4824-bbce-a998f9a0025c None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:31.378818 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-a4cb3a69-7f7e-4824-bbce-a998f9a0025c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:31.379474 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:31.399147 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:31.419409 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ab0fc0-a8d1-417a-8545-1f410977c80c None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:31.457664 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb05af21-63e0-4ad5-abca-d991642bb20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:31.458570 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb05af21-63e0-4ad5-abca-d991642bb20e None admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:31.458850 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb05af21-63e0-4ad5-abca-d991642bb20e None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:31.459125 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb05af21-63e0-4ad5-abca-d991642bb20e None admin] PATH_INFO: `/v3/projects/a44a5e0e3e2d4b8e9b9cec7b5b5c725c/users/28e59688485241c4b6d51f99bbb3d059/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:31.461348 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb05af21-63e0-4ad5-abca-d991642bb20e None admin] RBAC: Authorizing `identity:create_grant(project_id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c, user_id=28e59688485241c4b6d51f99bbb3d059, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:31.462830 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb05af21-63e0-4ad5-abca-d991642bb20e None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:31.478207 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 411/816] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jun 23 01:44:31 2022] PUT /identity/v3/projects/a44a5e0e3e2d4b8e9b9cec7b5b5c725c/users/28e59688485241c4b6d51f99bbb3d059/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:44:31.479040 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb05af21-63e0-4ad5-abca-d991642bb20e None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:31.574738 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 406/817] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:44:31.574738 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb3a69-7f7e-4824-bbce-a998f9a0025c None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:31.608732 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb05af21-63e0-4ad5-abca-d991642bb20e None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:31.636648 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:31.651479 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30954d4-dfd5-4907-a658-12d7763a454d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:31.652074 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30954d4-dfd5-4907-a658-12d7763a454d None admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:31.652412 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30954d4-dfd5-4907-a658-12d7763a454d None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:31.652696 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30954d4-dfd5-4907-a658-12d7763a454d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:31.653445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30954d4-dfd5-4907-a658-12d7763a454d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:31.655166 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30954d4-dfd5-4907-a658-12d7763a454d None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:31.675899 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 412/818] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Jun 23 01:44:31 2022] POST /identity/v3/endpoints => generated 327 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:44:31.677023 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30954d4-dfd5-4907-a658-12d7763a454d None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:31.698147 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb3a69-7f7e-4824-bbce-a998f9a0025c None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:31.760389 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950514d-7d45-4efc-a161-3d553bb6c9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:31.761138 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950514d-7d45-4efc-a161-3d553bb6c9ff None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:31.761468 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950514d-7d45-4efc-a161-3d553bb6c9ff None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:31.761748 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950514d-7d45-4efc-a161-3d553bb6c9ff None admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:31.762659 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3950514d-7d45-4efc-a161-3d553bb6c9ff None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:31.763122 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3950514d-7d45-4efc-a161-3d553bb6c9ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:31.765285 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3950514d-7d45-4efc-a161-3d553bb6c9ff None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:31.787958 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 407/819] 10.222.0.26 () {56 vars in 1133 bytes} [Thu Jun 23 01:44:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:31.788810 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950514d-7d45-4efc-a161-3d553bb6c9ff None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:31.802943 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:31.815903 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30954d4-dfd5-4907-a658-12d7763a454d None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:31.863177 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d519a-0de4-4053-85c9-c4bad60a7545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:31.864252 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d519a-0de4-4053-85c9-c4bad60a7545 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:31.864684 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d519a-0de4-4053-85c9-c4bad60a7545 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:31.865135 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d519a-0de4-4053-85c9-c4bad60a7545 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:31.873604 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815d519a-0de4-4053-85c9-c4bad60a7545 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:31.875073 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815d519a-0de4-4053-85c9-c4bad60a7545 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:31.889839 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-815d519a-0de4-4053-85c9-c4bad60a7545 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:31.889839 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:31.889839 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:31.889839 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:31.889839 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:31.889839 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:31.889839 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:31.889839 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:31.889839 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:31.889839 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:31.889839 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:31.889839 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:31.889839 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 01:44:31.889839 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 01:44:31.889839 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 01:44:31.890752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 01:44:31.890752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:31.890752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:31.890752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:31.890752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:31.890752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:31.890752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:31.890752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:31.890752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:31.890752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:31.890752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:31.890752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:31.890752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:31.890752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:31.890752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:31.890752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:31.890752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:31.891736 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 01:44:31.891736 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 01:44:31.891736 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 01:44:31.891736 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 01:44:31.891736 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 01:44:31.891736 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 01:44:31.891736 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:31.891736 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:31.895884 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 413/820] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:44:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:31.898693 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d519a-0de4-4053-85c9-c4bad60a7545 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:31.913944 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950514d-7d45-4efc-a161-3d553bb6c9ff None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:31.949021 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:31.958873 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced16ea7-04d9-45ae-b881-6d4811c3ef69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:31.960053 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced16ea7-04d9-45ae-b881-6d4811c3ef69 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:31.960334 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced16ea7-04d9-45ae-b881-6d4811c3ef69 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:31.960608 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced16ea7-04d9-45ae-b881-6d4811c3ef69 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:31.961329 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced16ea7-04d9-45ae-b881-6d4811c3ef69 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:31.962641 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced16ea7-04d9-45ae-b881-6d4811c3ef69 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:31.977608 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-ced16ea7-04d9-45ae-b881-6d4811c3ef69 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 01:44:31.977608 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:31.977608 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 01:44:31.977608 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 01:44:31.977608 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 01:44:31.977608 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 01:44:31.977608 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 01:44:31.977608 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:44:31.977608 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 01:44:31.977608 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:31.977608 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 01:44:31.977608 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:31.977608 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:31.977608 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:31.977608 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:31.977608 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:31.978584 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:31.978584 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:31.978584 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:31.978584 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:31.978584 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:31.978584 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:31.978584 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:31.978584 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 01:44:31.978584 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 01:44:31.978584 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 01:44:31.978584 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 01:44:31.978584 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:31.978584 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:31.978584 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:31.978584 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:31.978584 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:31.979457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:31.979457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:31.979457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:31.979457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:31.979457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:31.979457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:31.979457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:31.979457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:31.979457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:31.979457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:31.979457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:31.979457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 01:44:31.979457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:44:31.979457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 01:44:31.979457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:31.979457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 408/821] 10.222.0.26 () {56 vars in 1129 bytes} [Thu Jun 23 01:44:31 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:31.980302 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced16ea7-04d9-45ae-b881-6d4811c3ef69 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:32.025254 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d519a-0de4-4053-85c9-c4bad60a7545 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:32.059567 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c18a74-e44e-46d9-9289-dd274a51ecbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:32.060205 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c18a74-e44e-46d9-9289-dd274a51ecbc None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:32.060488 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c18a74-e44e-46d9-9289-dd274a51ecbc None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:32.060764 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c18a74-e44e-46d9-9289-dd274a51ecbc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:32.074331 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:32.107936 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced16ea7-04d9-45ae-b881-6d4811c3ef69 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:32.130563 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c18a74-e44e-46d9-9289-dd274a51ecbc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:32.132090 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c18a74-e44e-46d9-9289-dd274a51ecbc None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:32.144077 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf00f64-423e-4ae7-aa3a-a88d276ec8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:32.144812 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf00f64-423e-4ae7-aa3a-a88d276ec8e7 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:32.145101 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf00f64-423e-4ae7-aa3a-a88d276ec8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:32.145301 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf00f64-423e-4ae7-aa3a-a88d276ec8e7 None admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:32.146098 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf00f64-423e-4ae7-aa3a-a88d276ec8e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:32.146412 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf00f64-423e-4ae7-aa3a-a88d276ec8e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:32.147933 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf00f64-423e-4ae7-aa3a-a88d276ec8e7 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:32.166561 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 409/822] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jun 23 01:44:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:32.166561 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf00f64-423e-4ae7-aa3a-a88d276ec8e7 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:32.181785 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:32.245816 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 414/823] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:44:31 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 01:44:32.246738 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c18a74-e44e-46d9-9289-dd274a51ecbc None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:32.277716 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf00f64-423e-4ae7-aa3a-a88d276ec8e7 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:32.318957 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0546c45-08be-4870-832f-6f10ec5f129e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:32.319850 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0546c45-08be-4870-832f-6f10ec5f129e None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:32.320234 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0546c45-08be-4870-832f-6f10ec5f129e None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:32.320600 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0546c45-08be-4870-832f-6f10ec5f129e None admin] PATH_INFO: `/v3/services` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:32.321321 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0546c45-08be-4870-832f-6f10ec5f129e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:32.321631 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0546c45-08be-4870-832f-6f10ec5f129e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:32.323149 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0546c45-08be-4870-832f-6f10ec5f129e None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:32.325074 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:32.331728 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 410/824] 10.222.0.26 () {56 vars in 1124 bytes} [Thu Jun 23 01:44:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:32.332531 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0546c45-08be-4870-832f-6f10ec5f129e None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:32.360750 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c18a74-e44e-46d9-9289-dd274a51ecbc None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:32.396528 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34844db-ffe2-44b2-a29e-37927250f96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:32.397408 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34844db-ffe2-44b2-a29e-37927250f96d None admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:32.397691 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34844db-ffe2-44b2-a29e-37927250f96d None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:32.397966 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34844db-ffe2-44b2-a29e-37927250f96d None admin] PATH_INFO: `/v3/projects/a44a5e0e3e2d4b8e9b9cec7b5b5c725c/users/168e6181ad22460990f1f7361b483364/roles/c20e054f00164967a4c16e7500c93361` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:32.406009 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:32.413416 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34844db-ffe2-44b2-a29e-37927250f96d None admin] RBAC: Authorizing `identity:create_grant(project_id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c, user_id=168e6181ad22460990f1f7361b483364, role_id=c20e054f00164967a4c16e7500c93361)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:32.414898 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34844db-ffe2-44b2-a29e-37927250f96d None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:32.439250 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 415/825] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jun 23 01:44:32 2022] PUT /identity/v3/projects/a44a5e0e3e2d4b8e9b9cec7b5b5c725c/users/168e6181ad22460990f1f7361b483364/roles/c20e054f00164967a4c16e7500c93361 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:44:32.439250 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34844db-ffe2-44b2-a29e-37927250f96d None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:32.455480 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0546c45-08be-4870-832f-6f10ec5f129e None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:32.496201 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a49a60-e35e-4aa7-b039-ed870353632a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:32.497126 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a49a60-e35e-4aa7-b039-ed870353632a None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:32.497506 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a49a60-e35e-4aa7-b039-ed870353632a None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:32.497845 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a49a60-e35e-4aa7-b039-ed870353632a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:32.511151 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a49a60-e35e-4aa7-b039-ed870353632a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:32.513021 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a49a60-e35e-4aa7-b039-ed870353632a None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:32.533495 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-d7a49a60-e35e-4aa7-b039-ed870353632a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:32.533495 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:32.533495 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:32.533495 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:32.533495 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:32.533495 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:32.533495 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:32.533495 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:32.533495 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:32.533495 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:32.533495 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:32.533495 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:32.533495 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:32.533495 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:32.533495 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:32.534727 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:32.534727 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:32.534727 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:32.534727 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:32.534727 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:32.534727 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:32.534727 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:32.534727 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:32.534727 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:32.534727 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:32.534727 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:32.534727 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:32.534727 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:32.534727 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:32.534727 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:32.534727 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:32.534727 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:32.535689 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:32.535689 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:32.535689 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:32.535689 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:32.540377 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 411/826] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:32.541351 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a49a60-e35e-4aa7-b039-ed870353632a None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:32.576872 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34844db-ffe2-44b2-a29e-37927250f96d None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:32.601317 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:32.625794 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f52664-03e9-40b3-896b-069b52376289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:32.626467 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f52664-03e9-40b3-896b-069b52376289 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:32.626753 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f52664-03e9-40b3-896b-069b52376289 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:32.627026 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f52664-03e9-40b3-896b-069b52376289 None admin] PATH_INFO: `/v3/services` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:32.628146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f52664-03e9-40b3-896b-069b52376289 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:32.630293 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f52664-03e9-40b3-896b-069b52376289 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:32.630756 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f52664-03e9-40b3-896b-069b52376289 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:32.636966 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 416/827] 10.222.0.26 () {56 vars in 1124 bytes} [Thu Jun 23 01:44:32 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:32.642155 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f52664-03e9-40b3-896b-069b52376289 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:32.666017 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a49a60-e35e-4aa7-b039-ed870353632a None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:32.696678 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:32.713497 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4250547f-6a2f-4aac-bf7b-948c9be2dede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:32.714781 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4250547f-6a2f-4aac-bf7b-948c9be2dede None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:32.715300 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4250547f-6a2f-4aac-bf7b-948c9be2dede None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:32.716002 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4250547f-6a2f-4aac-bf7b-948c9be2dede None admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:32.717941 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4250547f-6a2f-4aac-bf7b-948c9be2dede None admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:32.718458 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4250547f-6a2f-4aac-bf7b-948c9be2dede None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:32.720110 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4250547f-6a2f-4aac-bf7b-948c9be2dede None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:32.741412 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 412/828] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:32.743171 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4250547f-6a2f-4aac-bf7b-948c9be2dede None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:32.754702 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f52664-03e9-40b3-896b-069b52376289 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:32.790033 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270da98a-18ba-41b7-8107-4fc6e5aef0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:32.790690 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270da98a-18ba-41b7-8107-4fc6e5aef0f7 None admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:32.790971 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270da98a-18ba-41b7-8107-4fc6e5aef0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:32.791280 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270da98a-18ba-41b7-8107-4fc6e5aef0f7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:32.792032 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270da98a-18ba-41b7-8107-4fc6e5aef0f7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:32.793406 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270da98a-18ba-41b7-8107-4fc6e5aef0f7 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:32.820031 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 417/829] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Jun 23 01:44:32 2022] POST /identity/v3/endpoints => generated 346 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 01:44:32.880649 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4250547f-6a2f-4aac-bf7b-948c9be2dede None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:32.907572 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:32.924717 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd3e93e-7cdc-4a12-a696-034a5d7b8487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:32.925531 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3e93e-7cdc-4a12-a696-034a5d7b8487 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:32.925810 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3e93e-7cdc-4a12-a696-034a5d7b8487 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:32.926083 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3e93e-7cdc-4a12-a696-034a5d7b8487 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:32.934287 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd3e93e-7cdc-4a12-a696-034a5d7b8487 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:32.935789 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd3e93e-7cdc-4a12-a696-034a5d7b8487 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:32.950978 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-5dd3e93e-7cdc-4a12-a696-034a5d7b8487 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:32.950978 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:32.950978 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:32.950978 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:32.950978 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:32.950978 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:32.950978 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:32.950978 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:32.950978 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:32.950978 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:32.950978 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:32.950978 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:32.950978 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 01:44:32.950978 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 01:44:32.950978 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 01:44:32.951945 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 01:44:32.951945 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:32.951945 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:32.951945 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:32.951945 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:32.951945 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:32.951945 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:32.951945 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:32.951945 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:32.951945 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:32.951945 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:32.951945 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:32.951945 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:32.951945 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:32.951945 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:32.951945 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:32.951945 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:32.953334 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 01:44:32.953334 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 01:44:32.953334 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 01:44:32.953334 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 01:44:32.953334 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 01:44:32.953334 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 01:44:32.953334 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:32.953334 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:32.953334 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 413/830] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:44:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:44:32.960415 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270da98a-18ba-41b7-8107-4fc6e5aef0f7 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:33.070829 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270da98a-18ba-41b7-8107-4fc6e5aef0f7 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:33.080732 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:33.108491 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ac4a4f-ce57-4363-a92b-2a8598c97047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:33.109505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ac4a4f-ce57-4363-a92b-2a8598c97047 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:33.110011 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ac4a4f-ce57-4363-a92b-2a8598c97047 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:33.110482 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ac4a4f-ce57-4363-a92b-2a8598c97047 None admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:33.111505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ac4a4f-ce57-4363-a92b-2a8598c97047 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:33.112079 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ac4a4f-ce57-4363-a92b-2a8598c97047 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:33.113732 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ac4a4f-ce57-4363-a92b-2a8598c97047 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:33.131393 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 418/831] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jun 23 01:44:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:33.133432 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd3e93e-7cdc-4a12-a696-034a5d7b8487 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:33.238056 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd3e93e-7cdc-4a12-a696-034a5d7b8487 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:33.272451 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b24ad8-c3ce-4d26-b7e2-f36f17a73504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:33.273129 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b24ad8-c3ce-4d26-b7e2-f36f17a73504 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:33.273413 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b24ad8-c3ce-4d26-b7e2-f36f17a73504 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:33.273688 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b24ad8-c3ce-4d26-b7e2-f36f17a73504 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:33.274532 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b24ad8-c3ce-4d26-b7e2-f36f17a73504 None admin] RBAC: Adding query filter params (role.id=7b20d65f49ee4a48894b6c060bb5d580, scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c, user.id=ae4e83df33b64a3ebf28f54e2d94fe63) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:33.274842 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b24ad8-c3ce-4d26-b7e2-f36f17a73504 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:33.276347 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b24ad8-c3ce-4d26-b7e2-f36f17a73504 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:33.282692 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 414/832] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:44:33 2022] GET /identity/v3/role_assignments?user.id=ae4e83df33b64a3ebf28f54e2d94fe63&scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c&role.id=7b20d65f49ee4a48894b6c060bb5d580 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:33.497275 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b85f31-460a-489e-81eb-886f51be38d0 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:33.497763 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b85f31-460a-489e-81eb-886f51be38d0 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:33.497763 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b85f31-460a-489e-81eb-886f51be38d0 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:33.499360 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 419/833] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:33.504749 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e535336-1f56-4b3c-a661-b2e4ec8dcb99 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:33.505158 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e535336-1f56-4b3c-a661-b2e4ec8dcb99 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:33.505342 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e535336-1f56-4b3c-a661-b2e4ec8dcb99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:33.521979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-3e535336-1f56-4b3c-a661-b2e4ec8dcb99 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:33.523154 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-3e535336-1f56-4b3c-a661-b2e4ec8dcb99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:33.549087 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:33.595815 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 415/834] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:33 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:44:33.619370 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b85f31-460a-489e-81eb-886f51be38d0 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:33.721909 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b85f31-460a-489e-81eb-886f51be38d0 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:33.756497 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f0cea9-47b6-4a07-8556-01c3498f8772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:33.757253 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0cea9-47b6-4a07-8556-01c3498f8772 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:33.757529 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0cea9-47b6-4a07-8556-01c3498f8772 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:33.757797 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0cea9-47b6-4a07-8556-01c3498f8772 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:33.768277 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f0cea9-47b6-4a07-8556-01c3498f8772 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:44:33.775983 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-51f0cea9-47b6-4a07-8556-01c3498f8772 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:44:33.775983 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:33.775983 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:33.775983 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:33.775983 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:33.775983 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:33.775983 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:33.775983 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:33.775983 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:33.775983 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:33.775983 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:33.775983 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:33.775983 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:44:33.775983 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:44:33.775983 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:44:33.775983 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:44:33.776902 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:44:33.776902 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:44:33.776902 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:44:33.776902 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:44:33.776902 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:33.776902 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:33.776902 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:33.776902 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:33.776902 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:33.776902 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:33.776902 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:33.776902 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:33.776902 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:33.776902 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:33.776902 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:33.776902 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:33.778144 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:33.778144 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:33.778144 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:33.778144 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:33.778144 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:44:33.778144 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:44:33.778144 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:44:33.778144 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:44:33.778144 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:44:33.778144 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:44:33.778144 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:44:33.778144 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:33.778144 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 420/835] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jun 23 01:44:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:33.783348 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e535336-1f56-4b3c-a661-b2e4ec8dcb99 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:33.880679 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e535336-1f56-4b3c-a661-b2e4ec8dcb99 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:33.915113 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb74a4-b9cd-410b-9886-eb5a8ed6e7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:33.915843 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb74a4-b9cd-410b-9886-eb5a8ed6e7bb None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:33.916125 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb74a4-b9cd-410b-9886-eb5a8ed6e7bb None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:33.916399 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb74a4-b9cd-410b-9886-eb5a8ed6e7bb None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:33.917153 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abb74a4-b9cd-410b-9886-eb5a8ed6e7bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:33.917468 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abb74a4-b9cd-410b-9886-eb5a8ed6e7bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:33.918828 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abb74a4-b9cd-410b-9886-eb5a8ed6e7bb None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:33.927841 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 416/836] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:44:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:33.933386 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f0cea9-47b6-4a07-8556-01c3498f8772 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:33.962292 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:34.034945 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f0cea9-47b6-4a07-8556-01c3498f8772 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:34.071580 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d8518-698b-423a-9c19-b7bbd1712f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:34.072377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d8518-698b-423a-9c19-b7bbd1712f0d None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:34.072655 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d8518-698b-423a-9c19-b7bbd1712f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:34.072927 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d8518-698b-423a-9c19-b7bbd1712f0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:34.084144 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2d8518-698b-423a-9c19-b7bbd1712f0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:34.085614 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2d8518-698b-423a-9c19-b7bbd1712f0d None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:34.102798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-4c2d8518-698b-423a-9c19-b7bbd1712f0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:34.102798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:34.102798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:34.102798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:34.102798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:34.102798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:34.102798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:34.102798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:34.102798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:34.102798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:34.102798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:34.102798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:34.102798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:34.102798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:34.102798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:34.103796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:34.103796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:34.103796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:34.103796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:34.103796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:34.103796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:34.103796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:34.103796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:34.103796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:34.103796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:34.103796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:34.103796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:34.103796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:34.103796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:34.103796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:34.103796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:34.103796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:34.104694 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:34.104694 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:34.104694 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:34.104694 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:34.105891 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 421/837] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:34.111897 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb74a4-b9cd-410b-9886-eb5a8ed6e7bb None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:34.137265 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:34.212175 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb74a4-b9cd-410b-9886-eb5a8ed6e7bb None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:34.250030 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eba7e5-33d6-40d4-89b0-f799336b3231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:34.250754 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eba7e5-33d6-40d4-89b0-f799336b3231 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:34.251033 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eba7e5-33d6-40d4-89b0-f799336b3231 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:34.251339 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eba7e5-33d6-40d4-89b0-f799336b3231 None admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:34.252062 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eba7e5-33d6-40d4-89b0-f799336b3231 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:34.252365 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eba7e5-33d6-40d4-89b0-f799336b3231 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:34.253845 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eba7e5-33d6-40d4-89b0-f799336b3231 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:34.266496 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 417/838] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:34.272296 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d8518-698b-423a-9c19-b7bbd1712f0d None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:34.353887 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:34.374565 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d8518-698b-423a-9c19-b7bbd1712f0d None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:34.408491 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e71c7-8509-46ae-9411-6293a1cd7e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:34.409361 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e71c7-8509-46ae-9411-6293a1cd7e65 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:34.409642 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e71c7-8509-46ae-9411-6293a1cd7e65 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:34.409917 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e71c7-8509-46ae-9411-6293a1cd7e65 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:34.417683 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367e71c7-8509-46ae-9411-6293a1cd7e65 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:34.419151 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367e71c7-8509-46ae-9411-6293a1cd7e65 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:34.433596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-367e71c7-8509-46ae-9411-6293a1cd7e65 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 01:44:34.433596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:34.433596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:34.433596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:34.433596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:34.433596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:34.433596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:34.433596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:34.433596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:34.433596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:34.433596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:34.433596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:34.433596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 01:44:34.433596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 01:44:34.433596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 01:44:34.433596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 01:44:34.434505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:34.434505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:34.434505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 01:44:34.434505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:34.434505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 01:44:34.434505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:34.434505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:34.434505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:34.434505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:34.434505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:34.434505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:34.434505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:34.434505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:34.434505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:34.434505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:34.434505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:34.434505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:34.434505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:34.435456 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:34.435456 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:34.435456 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 01:44:34.435456 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 01:44:34.435456 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 01:44:34.435456 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 01:44:34.435456 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 01:44:34.435456 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 01:44:34.435456 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 01:44:34.435456 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:34.435456 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 422/839] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jun 23 01:44:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:34.440430 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eba7e5-33d6-40d4-89b0-f799336b3231 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:34.519238 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae5cfb-55cb-476b-bfb2-cd037bf90f11 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:34.519238 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae5cfb-55cb-476b-bfb2-cd037bf90f11 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:34.519238 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae5cfb-55cb-476b-bfb2-cd037bf90f11 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:34.519238 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 423/840] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:34.525228 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e286948e-8d4a-4d25-83f9-c0d6587a06f7 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:34.525518 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e286948e-8d4a-4d25-83f9-c0d6587a06f7 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:34.525797 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e286948e-8d4a-4d25-83f9-c0d6587a06f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:34.545026 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:34.550960 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-e286948e-8d4a-4d25-83f9-c0d6587a06f7 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:34.553436 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-e286948e-8d4a-4d25-83f9-c0d6587a06f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:34.566088 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eba7e5-33d6-40d4-89b0-f799336b3231 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:34.604434 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97ae198-d612-498d-aa09-334c96bdeb5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:34.605141 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ae198-d612-498d-aa09-334c96bdeb5c None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:34.605423 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ae198-d612-498d-aa09-334c96bdeb5c None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:34.605699 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ae198-d612-498d-aa09-334c96bdeb5c None admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:34.606555 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97ae198-d612-498d-aa09-334c96bdeb5c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:34.606863 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97ae198-d612-498d-aa09-334c96bdeb5c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:34.608325 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97ae198-d612-498d-aa09-334c96bdeb5c None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:34.627131 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 418/841] 10.222.0.26 () {56 vars in 1135 bytes} [Thu Jun 23 01:44:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:44:34.632269 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97ae198-d612-498d-aa09-334c96bdeb5c None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:34.647372 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 424/842] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:34 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 01:44:34.671928 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e286948e-8d4a-4d25-83f9-c0d6587a06f7 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:34.746043 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:34.748925 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97ae198-d612-498d-aa09-334c96bdeb5c None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:34.779095 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e286948e-8d4a-4d25-83f9-c0d6587a06f7 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:34.786383 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ed075-bcfc-4546-96ed-4e27e13366d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:34.787334 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ed075-bcfc-4546-96ed-4e27e13366d6 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:34.787896 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ed075-bcfc-4546-96ed-4e27e13366d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:34.788372 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ed075-bcfc-4546-96ed-4e27e13366d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:34.800359 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:34.817994 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99726f2f-a68b-4c51-aad4-cd9ba709343f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:34.818728 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99726f2f-a68b-4c51-aad4-cd9ba709343f None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:34.819014 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99726f2f-a68b-4c51-aad4-cd9ba709343f None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:34.819321 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99726f2f-a68b-4c51-aad4-cd9ba709343f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:34.831093 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99726f2f-a68b-4c51-aad4-cd9ba709343f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:44:34.839597 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-99726f2f-a68b-4c51-aad4-cd9ba709343f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 01:44:34.839597 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:34.839597 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:34.839597 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:34.839597 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:34.839597 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:34.839597 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:34.839597 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:34.839597 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:34.839597 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:34.839597 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:34.839597 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:34.839597 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:44:34.839597 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:44:34.839597 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:44:34.839597 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:44:34.840552 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:44:34.840552 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:44:34.840552 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:44:34.840552 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:44:34.840552 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:34.840552 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:34.840552 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:34.840552 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:34.840552 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:34.840552 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:34.840552 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:34.840552 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:34.840552 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:34.840552 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:34.840552 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:34.840552 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:34.841378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:34.841378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:34.841378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:34.841378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:34.841378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:44:34.841378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:44:34.841378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:44:34.841378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:44:34.841378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:44:34.841378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:44:34.841378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 01:44:34.841378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:34.845808 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 425/843] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jun 23 01:44:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:44:34.846735 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99726f2f-a68b-4c51-aad4-cd9ba709343f None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:34.868141 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6ed075-bcfc-4546-96ed-4e27e13366d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:34.869903 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6ed075-bcfc-4546-96ed-4e27e13366d6 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:34.955800 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99726f2f-a68b-4c51-aad4-cd9ba709343f None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:34.978189 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 419/844] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:44:34 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 23 01:44:34.987263 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ed075-bcfc-4546-96ed-4e27e13366d6 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:34.989995 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:34.997268 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc92ad1e-f5b6-44d5-9eec-03773d3c2f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:34.997850 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc92ad1e-f5b6-44d5-9eec-03773d3c2f28 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:34.998133 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc92ad1e-f5b6-44d5-9eec-03773d3c2f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:34.998412 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc92ad1e-f5b6-44d5-9eec-03773d3c2f28 None admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:34.999182 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc92ad1e-f5b6-44d5-9eec-03773d3c2f28 None admin] RBAC: Adding query filter params (name=service) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:34.999593 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc92ad1e-f5b6-44d5-9eec-03773d3c2f28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:35.001277 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc92ad1e-f5b6-44d5-9eec-03773d3c2f28 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:35.015713 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 426/845] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Jun 23 01:44:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:35.016818 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc92ad1e-f5b6-44d5-9eec-03773d3c2f28 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:35.028848 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:35.089023 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ed075-bcfc-4546-96ed-4e27e13366d6 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:35.124289 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e7b8c-1025-49a1-bc8b-c39f1ab817f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:35.125102 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e7b8c-1025-49a1-bc8b-c39f1ab817f8 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:35.125384 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e7b8c-1025-49a1-bc8b-c39f1ab817f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:35.125657 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e7b8c-1025-49a1-bc8b-c39f1ab817f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:35.126721 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc92ad1e-f5b6-44d5-9eec-03773d3c2f28 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:35.136255 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92e7b8c-1025-49a1-bc8b-c39f1ab817f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:35.137704 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92e7b8c-1025-49a1-bc8b-c39f1ab817f8 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:35.155694 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-f92e7b8c-1025-49a1-bc8b-c39f1ab817f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:35.155694 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:35.155694 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:35.155694 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:35.155694 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:35.155694 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:35.155694 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:35.155694 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:35.155694 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:35.155694 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:35.155694 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:35.155694 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:35.155694 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:35.155694 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:35.155694 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:35.156606 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:35.156606 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:35.156606 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:35.156606 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:35.156606 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:35.156606 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:35.156606 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:35.156606 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:35.156606 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:35.156606 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:35.156606 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:35.156606 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:35.156606 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:35.156606 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:35.156606 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:35.156606 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:35.156606 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:35.157492 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:35.157492 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:35.157492 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:35.157492 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:35.157492 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 420/846] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:44:35.158606 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231dab4d-44fa-4b80-a8b9-493884d6d3bc None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:35.159103 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231dab4d-44fa-4b80-a8b9-493884d6d3bc None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:35.159587 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231dab4d-44fa-4b80-a8b9-493884d6d3bc None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:35.165752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 421/847] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:35.165752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231dab4d-44fa-4b80-a8b9-493884d6d3bc None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:35.171559 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b26ad3-35eb-4c16-9d53-874aecba7584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:35.172766 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b26ad3-35eb-4c16-9d53-874aecba7584 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:35.173051 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b26ad3-35eb-4c16-9d53-874aecba7584 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:35.173324 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b26ad3-35eb-4c16-9d53-874aecba7584 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:35.185686 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b26ad3-35eb-4c16-9d53-874aecba7584 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:35.187167 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b26ad3-35eb-4c16-9d53-874aecba7584 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:35.205421 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-f7b26ad3-35eb-4c16-9d53-874aecba7584 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:35.205421 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:35.205421 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:35.205421 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:35.205421 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:35.205421 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:35.205421 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:35.205421 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:35.205421 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:35.205421 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:35.205421 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:35.205421 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:35.205421 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:35.205421 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:35.205421 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:35.206366 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:35.206366 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:35.206366 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:35.206366 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:35.206366 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:35.206366 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:35.206366 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:35.206366 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:35.206366 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:35.206366 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:35.206366 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:35.206366 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:35.206366 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:35.206366 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:35.206366 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:35.206366 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:35.206366 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:35.208120 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:35.208120 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:35.208120 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:35.208120 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:35.208120 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 427/848] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:44:35.208120 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880e6a0c-7d48-4795-b90a-5dcacb7c141f None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:35.208466 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880e6a0c-7d48-4795-b90a-5dcacb7c141f None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:35.208676 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880e6a0c-7d48-4795-b90a-5dcacb7c141f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:35.227883 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-880e6a0c-7d48-4795-b90a-5dcacb7c141f None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:35.229035 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-880e6a0c-7d48-4795-b90a-5dcacb7c141f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:35.270431 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:35.271291 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:35.283300 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231dab4d-44fa-4b80-a8b9-493884d6d3bc None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:35.315232 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 428/849] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:35 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:44:35.315232 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880e6a0c-7d48-4795-b90a-5dcacb7c141f None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:35.322166 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b0b6f-fc6e-4ff5-8052-54a0718f1fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:35.322997 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b0b6f-fc6e-4ff5-8052-54a0718f1fa8 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:35.323305 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b0b6f-fc6e-4ff5-8052-54a0718f1fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:35.323582 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b0b6f-fc6e-4ff5-8052-54a0718f1fa8 None admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:35.324312 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898b0b6f-fc6e-4ff5-8052-54a0718f1fa8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:35.324618 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898b0b6f-fc6e-4ff5-8052-54a0718f1fa8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:35.325996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898b0b6f-fc6e-4ff5-8052-54a0718f1fa8 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:35.344509 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 422/850] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:35.344509 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b0b6f-fc6e-4ff5-8052-54a0718f1fa8 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:35.420604 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880e6a0c-7d48-4795-b90a-5dcacb7c141f None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:35.448488 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b0b6f-fc6e-4ff5-8052-54a0718f1fa8 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:35.455711 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade601ce-5c07-4aa9-affc-34fb22833b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:35.456376 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade601ce-5c07-4aa9-affc-34fb22833b31 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:35.456654 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade601ce-5c07-4aa9-affc-34fb22833b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:35.456927 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade601ce-5c07-4aa9-affc-34fb22833b31 None admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:35.457651 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade601ce-5c07-4aa9-affc-34fb22833b31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:35.457954 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade601ce-5c07-4aa9-affc-34fb22833b31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:35.459373 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade601ce-5c07-4aa9-affc-34fb22833b31 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:35.470519 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 429/851] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:35.471383 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade601ce-5c07-4aa9-affc-34fb22833b31 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:35.489524 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32ce52-091e-44c2-9920-c5ae59cab92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:35.490359 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32ce52-091e-44c2-9920-c5ae59cab92e None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:35.490624 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32ce52-091e-44c2-9920-c5ae59cab92e None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:35.490900 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32ce52-091e-44c2-9920-c5ae59cab92e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:35.491598 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb32ce52-091e-44c2-9920-c5ae59cab92e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:35.493226 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb32ce52-091e-44c2-9920-c5ae59cab92e None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:35.507007 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:35.508867 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-fb32ce52-091e-44c2-9920-c5ae59cab92e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 01:44:35.508867 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:35.508867 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 01:44:35.508867 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 01:44:35.508867 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 01:44:35.508867 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 01:44:35.508867 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 01:44:35.508867 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:44:35.508867 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 01:44:35.508867 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:35.508867 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 01:44:35.508867 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:35.508867 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:35.508867 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:35.508867 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:35.508867 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:35.508867 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:35.510017 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:35.510017 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:35.510017 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:35.510017 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:35.510017 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:35.510017 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:35.510017 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 01:44:35.510017 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 01:44:35.510017 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 01:44:35.510017 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 01:44:35.510017 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:35.510017 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:35.510017 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:35.510017 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:35.510017 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:35.510017 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:35.510881 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:35.510881 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:35.510881 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:35.510881 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:35.510881 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:35.510881 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:35.510881 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:35.510881 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:35.510881 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:35.510881 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:35.510881 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 01:44:35.510881 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:44:35.510881 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 01:44:35.510881 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:35.510881 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 423/852] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Jun 23 01:44:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:35.511709 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32ce52-091e-44c2-9920-c5ae59cab92e None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:35.534746 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:35.579945 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade601ce-5c07-4aa9-affc-34fb22833b31 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:35.615482 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2241072-b153-42d7-b337-1ddef07edb52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:35.616210 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2241072-b153-42d7-b337-1ddef07edb52 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:35.616491 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2241072-b153-42d7-b337-1ddef07edb52 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:35.616768 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2241072-b153-42d7-b337-1ddef07edb52 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:35.616863 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32ce52-091e-44c2-9920-c5ae59cab92e None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:35.623664 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2241072-b153-42d7-b337-1ddef07edb52 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:35.625142 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2241072-b153-42d7-b337-1ddef07edb52 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:35.639588 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-d2241072-b153-42d7-b337-1ddef07edb52 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:35.639588 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:35.639588 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:35.639588 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:35.639588 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:35.639588 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:35.639588 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:35.639588 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:35.639588 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:35.639588 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:35.639588 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:35.639588 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:35.639588 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 01:44:35.639588 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 01:44:35.639588 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 01:44:35.640511 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 01:44:35.640511 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:35.640511 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:35.640511 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:35.640511 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:35.640511 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:35.640511 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:35.640511 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:35.640511 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:35.640511 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:35.640511 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:35.640511 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:35.640511 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:35.640511 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:35.640511 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:35.640511 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:35.640511 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:35.641381 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 01:44:35.641381 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 01:44:35.641381 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 01:44:35.641381 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 01:44:35.641381 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 01:44:35.641381 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 01:44:35.641381 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:35.641381 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:35.641381 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 430/853] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:44:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:35.642133 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2616b3-b06a-42d2-8e43-75b45d7854e0 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:35.642417 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2616b3-b06a-42d2-8e43-75b45d7854e0 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:35.642704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2616b3-b06a-42d2-8e43-75b45d7854e0 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:35.644101 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 431/854] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:35.648814 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2616b3-b06a-42d2-8e43-75b45d7854e0 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:35.652707 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbe66e2-3b15-4012-960c-c766795bad8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:35.653608 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe66e2-3b15-4012-960c-c766795bad8b None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:35.653883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe66e2-3b15-4012-960c-c766795bad8b None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:35.654152 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe66e2-3b15-4012-960c-c766795bad8b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:35.661858 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbe66e2-3b15-4012-960c-c766795bad8b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:35.663309 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbe66e2-3b15-4012-960c-c766795bad8b None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:35.677502 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-7cbe66e2-3b15-4012-960c-c766795bad8b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 01:44:35.677502 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:35.677502 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:35.677502 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:35.677502 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:35.677502 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:35.677502 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:35.677502 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:35.677502 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:35.677502 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:35.677502 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:35.677502 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:35.677502 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 01:44:35.677502 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 01:44:35.677502 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 01:44:35.677502 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 01:44:35.678443 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:35.678443 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:35.678443 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 01:44:35.678443 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:35.678443 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 01:44:35.678443 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:35.678443 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:35.678443 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:35.678443 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:35.678443 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:35.678443 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:35.678443 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:35.678443 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:35.678443 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:35.678443 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:35.678443 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:35.678443 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:35.678443 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:35.679421 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:35.679421 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:35.679421 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 01:44:35.679421 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 01:44:35.679421 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 01:44:35.679421 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 01:44:35.679421 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 01:44:35.679421 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 01:44:35.679421 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 01:44:35.679421 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:35.679421 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 424/855] 10.222.0.26 () {56 vars in 1117 bytes} [Thu Jun 23 01:44:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:35.680335 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbe66e2-3b15-4012-960c-c766795bad8b None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:35.746173 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:35.753862 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2616b3-b06a-42d2-8e43-75b45d7854e0 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:35.757752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:35.781507 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbe66e2-3b15-4012-960c-c766795bad8b None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:35.788855 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dedc46-9bfd-4597-91a3-ae28b982a560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:35.789505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dedc46-9bfd-4597-91a3-ae28b982a560 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:35.789782 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dedc46-9bfd-4597-91a3-ae28b982a560 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:35.790051 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dedc46-9bfd-4597-91a3-ae28b982a560 None admin] PATH_INFO: `/v3/services` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:35.790751 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dedc46-9bfd-4597-91a3-ae28b982a560 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:35.791048 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dedc46-9bfd-4597-91a3-ae28b982a560 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:35.792481 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dedc46-9bfd-4597-91a3-ae28b982a560 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:35.796876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 432/856] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Jun 23 01:44:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:35.798191 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fffb46a-69a4-4f14-bd8b-f5e7a5177c53 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:35.798482 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fffb46a-69a4-4f14-bd8b-f5e7a5177c53 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:35.798759 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fffb46a-69a4-4f14-bd8b-f5e7a5177c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:35.819750 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767d643-787b-4cf9-ac3c-0c76a2b918fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:35.820441 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767d643-787b-4cf9-ac3c-0c76a2b918fd None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:35.820723 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767d643-787b-4cf9-ac3c-0c76a2b918fd None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:35.820997 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767d643-787b-4cf9-ac3c-0c76a2b918fd None admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:35.821806 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7767d643-787b-4cf9-ac3c-0c76a2b918fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:35.822116 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7767d643-787b-4cf9-ac3c-0c76a2b918fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:35.823623 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7767d643-787b-4cf9-ac3c-0c76a2b918fd None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:35.837684 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 425/857] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jun 23 01:44:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:35.838570 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767d643-787b-4cf9-ac3c-0c76a2b918fd None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:35.932562 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-9fffb46a-69a4-4f14-bd8b-f5e7a5177c53 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:35.933806 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-9fffb46a-69a4-4f14-bd8b-f5e7a5177c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:35.953321 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767d643-787b-4cf9-ac3c-0c76a2b918fd None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:35.989120 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11db3b64-0017-4f20-8d59-bbb1e71c095a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:35.989796 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db3b64-0017-4f20-8d59-bbb1e71c095a None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:35.990078 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db3b64-0017-4f20-8d59-bbb1e71c095a None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:35.990350 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db3b64-0017-4f20-8d59-bbb1e71c095a None admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:35.991195 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11db3b64-0017-4f20-8d59-bbb1e71c095a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:35.991529 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11db3b64-0017-4f20-8d59-bbb1e71c095a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:35.992957 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11db3b64-0017-4f20-8d59-bbb1e71c095a None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:35.995919 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:36.004175 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 433/858] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:35 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:44:36.005002 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fffb46a-69a4-4f14-bd8b-f5e7a5177c53 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:36.010888 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 426/859] 10.222.0.26 () {56 vars in 1149 bytes} [Thu Jun 23 01:44:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:36.011577 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11db3b64-0017-4f20-8d59-bbb1e71c095a None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:36.091844 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:36.104738 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fffb46a-69a4-4f14-bd8b-f5e7a5177c53 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:36.110399 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11db3b64-0017-4f20-8d59-bbb1e71c095a None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:36.140241 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212b1de-de98-4d6d-9626-c659aaedee32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:36.140909 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212b1de-de98-4d6d-9626-c659aaedee32 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:36.141191 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212b1de-de98-4d6d-9626-c659aaedee32 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:36.141462 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212b1de-de98-4d6d-9626-c659aaedee32 None admin] PATH_INFO: `/v3/services` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:36.142179 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c212b1de-de98-4d6d-9626-c659aaedee32 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:36.142486 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c212b1de-de98-4d6d-9626-c659aaedee32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:36.143992 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c212b1de-de98-4d6d-9626-c659aaedee32 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:36.146114 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472cd01d-7fbb-4e04-8f9c-b3fc5256914c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:36.146773 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472cd01d-7fbb-4e04-8f9c-b3fc5256914c None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:36.147052 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472cd01d-7fbb-4e04-8f9c-b3fc5256914c None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:36.147365 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472cd01d-7fbb-4e04-8f9c-b3fc5256914c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:36.148204 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472cd01d-7fbb-4e04-8f9c-b3fc5256914c None admin] RBAC: Adding query filter params (role.id=7b20d65f49ee4a48894b6c060bb5d580, scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c, user.id=28e59688485241c4b6d51f99bbb3d059) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:36.148516 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472cd01d-7fbb-4e04-8f9c-b3fc5256914c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:36.148703 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 434/860] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Jun 23 01:44:36 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:36.149543 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212b1de-de98-4d6d-9626-c659aaedee32 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:36.149941 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472cd01d-7fbb-4e04-8f9c-b3fc5256914c None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:36.161256 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 427/861] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:44:36 2022] GET /identity/v3/role_assignments?user.id=28e59688485241c4b6d51f99bbb3d059&scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c&role.id=7b20d65f49ee4a48894b6c060bb5d580 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:36.161988 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472cd01d-7fbb-4e04-8f9c-b3fc5256914c None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:36.243838 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:36.300509 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472cd01d-7fbb-4e04-8f9c-b3fc5256914c None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:36.312617 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212b1de-de98-4d6d-9626-c659aaedee32 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:36.341965 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5de739-afc9-4b17-815a-80f4b60bed3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:36.342792 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5de739-afc9-4b17-815a-80f4b60bed3f None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:36.343078 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5de739-afc9-4b17-815a-80f4b60bed3f None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:36.343422 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5de739-afc9-4b17-815a-80f4b60bed3f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:36.344052 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5de739-afc9-4b17-815a-80f4b60bed3f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:36.345437 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5de739-afc9-4b17-815a-80f4b60bed3f None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:36.354553 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee4d43b-ec1c-4d90-84e9-2a247ffb287c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:36.355528 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4d43b-ec1c-4d90-84e9-2a247ffb287c None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:36.356081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4d43b-ec1c-4d90-84e9-2a247ffb287c None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:36.356541 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4d43b-ec1c-4d90-84e9-2a247ffb287c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:36.359273 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-ac5de739-afc9-4b17-815a-80f4b60bed3f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 01:44:36.359273 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:36.359273 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 01:44:36.359273 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 01:44:36.359273 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 01:44:36.359273 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 01:44:36.359273 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 01:44:36.359273 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:44:36.359273 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 01:44:36.359273 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:36.359273 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 01:44:36.359273 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:36.359273 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:36.359273 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:36.359273 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:36.359273 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:36.360199 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:36.360199 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:36.360199 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:36.360199 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:36.360199 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:36.360199 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:36.360199 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:36.360199 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 01:44:36.360199 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 01:44:36.360199 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 01:44:36.360199 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 01:44:36.360199 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:36.360199 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:36.360199 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:36.360199 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:36.360199 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:36.361029 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:36.361029 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:36.361029 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:36.361029 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:36.361029 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:36.361029 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:36.361029 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:36.361029 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:36.361029 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:36.361029 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:36.361029 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:36.361029 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 01:44:36.361029 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:44:36.361029 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 01:44:36.361029 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:36.362290 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 428/862] 10.222.0.26 () {56 vars in 1132 bytes} [Thu Jun 23 01:44:36 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:36.368004 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5de739-afc9-4b17-815a-80f4b60bed3f None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:36.381420 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:36.433407 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee4d43b-ec1c-4d90-84e9-2a247ffb287c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:36.434993 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee4d43b-ec1c-4d90-84e9-2a247ffb287c None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:36.486029 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5de739-afc9-4b17-815a-80f4b60bed3f None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:36.515439 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:36.537136 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517de8b6-abc4-4cd9-bb45-dae31c237a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:36.538293 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517de8b6-abc4-4cd9-bb45-dae31c237a2f None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:36.539121 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517de8b6-abc4-4cd9-bb45-dae31c237a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:36.539750 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517de8b6-abc4-4cd9-bb45-dae31c237a2f None admin] PATH_INFO: `/v3/services` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:36.540767 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517de8b6-abc4-4cd9-bb45-dae31c237a2f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:36.541461 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517de8b6-abc4-4cd9-bb45-dae31c237a2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:36.543740 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517de8b6-abc4-4cd9-bb45-dae31c237a2f None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:36.550379 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 435/863] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:44:36 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 01:44:36.551473 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee4d43b-ec1c-4d90-84e9-2a247ffb287c None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:36.553864 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 429/864] 10.222.0.26 () {56 vars in 1126 bytes} [Thu Jun 23 01:44:36 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:44:36.564007 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517de8b6-abc4-4cd9-bb45-dae31c237a2f None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:36.617766 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:36.668564 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee4d43b-ec1c-4d90-84e9-2a247ffb287c None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:36.685985 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517de8b6-abc4-4cd9-bb45-dae31c237a2f None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:36.714691 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28daa2b2-c2e3-4f16-ae09-e7a208649c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:36.716422 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28daa2b2-c2e3-4f16-ae09-e7a208649c59 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:36.716854 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28daa2b2-c2e3-4f16-ae09-e7a208649c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:36.717305 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28daa2b2-c2e3-4f16-ae09-e7a208649c59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:36.723730 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c80a2c8-dcde-4a8a-b314-3a5b4b8f7ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:36.724557 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80a2c8-dcde-4a8a-b314-3a5b4b8f7ef2 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:36.725066 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80a2c8-dcde-4a8a-b314-3a5b4b8f7ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:36.725443 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80a2c8-dcde-4a8a-b314-3a5b4b8f7ef2 None admin] PATH_INFO: `/v3/services` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:36.726404 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c80a2c8-dcde-4a8a-b314-3a5b4b8f7ef2 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:36.726763 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c80a2c8-dcde-4a8a-b314-3a5b4b8f7ef2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:36.728438 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c80a2c8-dcde-4a8a-b314-3a5b4b8f7ef2 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:36.731084 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28daa2b2-c2e3-4f16-ae09-e7a208649c59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:36.733013 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28daa2b2-c2e3-4f16-ae09-e7a208649c59 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:36.739717 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 430/865] 10.222.0.26 () {56 vars in 1126 bytes} [Thu Jun 23 01:44:36 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:36.753442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-28daa2b2-c2e3-4f16-ae09-e7a208649c59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:36.753442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:36.753442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:36.753442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:36.753442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:36.753442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:36.753442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:36.753442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:36.753442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:36.753442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:36.753442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:36.753442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:36.753442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:36.753442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:36.753442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:36.754612 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:36.754612 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:36.754612 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:36.754612 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:36.754612 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:36.754612 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:36.754612 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:36.754612 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:36.754612 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:36.754612 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:36.754612 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:36.754612 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:36.754612 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:36.754612 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:36.754612 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:36.754612 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:36.754612 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:36.755752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:36.755752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:36.755752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:36.755752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:36.757745 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 436/866] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:36.763718 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c80a2c8-dcde-4a8a-b314-3a5b4b8f7ef2 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:36.822493 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:36.880397 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c80a2c8-dcde-4a8a-b314-3a5b4b8f7ef2 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:36.917199 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe00a6-6a95-46ee-a17a-3aaa8c0b78b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:36.918163 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe00a6-6a95-46ee-a17a-3aaa8c0b78b4 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:36.918593 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe00a6-6a95-46ee-a17a-3aaa8c0b78b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:36.919016 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe00a6-6a95-46ee-a17a-3aaa8c0b78b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:36.919949 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfe00a6-6a95-46ee-a17a-3aaa8c0b78b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:36.920411 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfe00a6-6a95-46ee-a17a-3aaa8c0b78b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:36.921946 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfe00a6-6a95-46ee-a17a-3aaa8c0b78b4 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:36.936462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 431/867] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:36.942338 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28daa2b2-c2e3-4f16-ae09-e7a208649c59 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:37.046207 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28daa2b2-c2e3-4f16-ae09-e7a208649c59 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:37.049582 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:37.081894 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce4a9ff-b874-49e4-8c95-6bf459319a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:37.082924 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4a9ff-b874-49e4-8c95-6bf459319a75 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:37.083383 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4a9ff-b874-49e4-8c95-6bf459319a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:37.083798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4a9ff-b874-49e4-8c95-6bf459319a75 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:37.090880 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce4a9ff-b874-49e4-8c95-6bf459319a75 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:37.092519 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce4a9ff-b874-49e4-8c95-6bf459319a75 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:37.107257 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-3ce4a9ff-b874-49e4-8c95-6bf459319a75 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:37.107257 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:37.107257 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:37.107257 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:37.107257 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:37.107257 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:37.107257 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:37.107257 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:37.107257 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:37.107257 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:37.107257 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:37.107257 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:37.107257 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 01:44:37.107257 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 01:44:37.107257 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 01:44:37.108340 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 01:44:37.108340 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:37.108340 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:37.108340 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:37.108340 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:37.108340 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:37.108340 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:37.108340 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:37.108340 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:37.108340 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:37.108340 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:37.108340 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:37.108340 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:37.108340 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:37.108340 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:37.108340 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:37.108340 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:37.109441 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 01:44:37.109441 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 01:44:37.109441 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 01:44:37.109441 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 01:44:37.109441 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 01:44:37.109441 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 01:44:37.109441 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:37.109441 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:37.111540 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 437/868] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:44:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:37.117651 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe00a6-6a95-46ee-a17a-3aaa8c0b78b4 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:37.231566 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe00a6-6a95-46ee-a17a-3aaa8c0b78b4 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:37.243485 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:37.269529 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b7e8e0-7e2c-4b91-a392-0baa3c19bfc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:37.270545 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7e8e0-7e2c-4b91-a392-0baa3c19bfc9 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:37.271011 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7e8e0-7e2c-4b91-a392-0baa3c19bfc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:37.271484 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7e8e0-7e2c-4b91-a392-0baa3c19bfc9 None admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:37.272502 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b7e8e0-7e2c-4b91-a392-0baa3c19bfc9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:37.272991 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b7e8e0-7e2c-4b91-a392-0baa3c19bfc9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:37.274624 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b7e8e0-7e2c-4b91-a392-0baa3c19bfc9 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:37.290556 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 432/869] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jun 23 01:44:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:37.296966 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce4a9ff-b874-49e4-8c95-6bf459319a75 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:37.403854 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce4a9ff-b874-49e4-8c95-6bf459319a75 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:37.440848 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbdb8cc-5911-4052-83bd-2ace49969794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:37.441525 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbdb8cc-5911-4052-83bd-2ace49969794 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:37.441804 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbdb8cc-5911-4052-83bd-2ace49969794 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:37.442079 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbdb8cc-5911-4052-83bd-2ace49969794 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:37.442942 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbdb8cc-5911-4052-83bd-2ace49969794 None admin] RBAC: Adding query filter params (role.id=c20e054f00164967a4c16e7500c93361, scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c, user.id=168e6181ad22460990f1f7361b483364) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:37.443273 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbdb8cc-5911-4052-83bd-2ace49969794 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:37.444765 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbdb8cc-5911-4052-83bd-2ace49969794 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:37.451122 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 438/870] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:44:37 2022] GET /identity/v3/role_assignments?user.id=168e6181ad22460990f1f7361b483364&scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c&role.id=c20e054f00164967a4c16e7500c93361 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:38.208855 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9e506-19a7-4649-862a-0c8bdabc900a None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:38.209375 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9e506-19a7-4649-862a-0c8bdabc900a None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:38.209375 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9e506-19a7-4649-862a-0c8bdabc900a None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:38.210901 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 433/871] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:38.216460 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d6b91-42f1-4260-bf6d-2fe3844d5728 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:38.217300 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d6b91-42f1-4260-bf6d-2fe3844d5728 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:38.217780 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d6b91-42f1-4260-bf6d-2fe3844d5728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:38.234958 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-e12d6b91-42f1-4260-bf6d-2fe3844d5728 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:38.236491 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-e12d6b91-42f1-4260-bf6d-2fe3844d5728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:38.262662 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:38.309340 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 439/872] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:38 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 01:44:38.332052 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e9e506-19a7-4649-862a-0c8bdabc900a None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:38.366132 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465aba9-584a-4002-bc2c-003fee7e6f56 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:38.366532 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465aba9-584a-4002-bc2c-003fee7e6f56 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:38.366907 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c465aba9-584a-4002-bc2c-003fee7e6f56 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:38.369833 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 440/873] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 01:44:38.373864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5787a3-f579-4079-9ea5-7e7026bcd1d2 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:38.375483 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5787a3-f579-4079-9ea5-7e7026bcd1d2 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:38.375760 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5787a3-f579-4079-9ea5-7e7026bcd1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:38.395069 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-7b5787a3-f579-4079-9ea5-7e7026bcd1d2 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:38.396454 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-7b5787a3-f579-4079-9ea5-7e7026bcd1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:38.442882 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e9e506-19a7-4649-862a-0c8bdabc900a None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:38.481112 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dcd3d3-b850-4dd7-b85f-e7a4845a53e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:38.481702 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dcd3d3-b850-4dd7-b85f-e7a4845a53e2 None admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:38.481984 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dcd3d3-b850-4dd7-b85f-e7a4845a53e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:38.482256 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dcd3d3-b850-4dd7-b85f-e7a4845a53e2 None admin] PATH_INFO: `/v3/services` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:38.482997 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dcd3d3-b850-4dd7-b85f-e7a4845a53e2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:38.484397 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dcd3d3-b850-4dd7-b85f-e7a4845a53e2 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:38.491094 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 441/874] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:38 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 01:44:38.507946 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 434/875] 10.222.0.26 () {60 vars in 1141 bytes} [Thu Jun 23 01:44:38 2022] POST /identity/v3/services => generated 251 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:44:38.513003 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5787a3-f579-4079-9ea5-7e7026bcd1d2 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:38.617101 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:38.631095 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5787a3-f579-4079-9ea5-7e7026bcd1d2 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:38.678618 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21371a6f-1227-4b44-855b-62fef7649ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:38.679462 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21371a6f-1227-4b44-855b-62fef7649ffa None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:38.680056 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21371a6f-1227-4b44-855b-62fef7649ffa None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:38.680327 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21371a6f-1227-4b44-855b-62fef7649ffa None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:38.680986 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21371a6f-1227-4b44-855b-62fef7649ffa None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:38.682455 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21371a6f-1227-4b44-855b-62fef7649ffa None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:38.695277 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-21371a6f-1227-4b44-855b-62fef7649ffa None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 01:44:38.695277 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:38.695277 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 01:44:38.695277 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 01:44:38.695277 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 01:44:38.695277 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 01:44:38.695277 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 01:44:38.695277 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:44:38.695277 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 01:44:38.695277 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:38.695277 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 01:44:38.695277 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:38.695277 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:38.695277 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:38.695277 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:38.695277 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:38.695277 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:38.696276 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:38.696276 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:38.696276 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:38.696276 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:38.696276 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:38.696276 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:38.696276 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 01:44:38.696276 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 01:44:38.696276 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 01:44:38.696276 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 01:44:38.696276 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:38.696276 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:38.696276 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:38.696276 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:38.696276 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:38.696276 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:38.697113 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:38.697113 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:38.697113 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:38.697113 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:38.697113 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:38.697113 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:38.697113 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:38.697113 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:38.697113 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:38.697113 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:38.697113 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 01:44:38.697113 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:44:38.697113 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 01:44:38.697113 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:38.700162 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 442/876] 10.222.0.26 () {56 vars in 1126 bytes} [Thu Jun 23 01:44:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:38.702975 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dcd3d3-b850-4dd7-b85f-e7a4845a53e2 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:38.737336 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:38.811669 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dcd3d3-b850-4dd7-b85f-e7a4845a53e2 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:38.824502 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f1fdb-0176-4048-aef8-c15b55f95ff1 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:38.824803 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f1fdb-0176-4048-aef8-c15b55f95ff1 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:38.825050 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f1fdb-0176-4048-aef8-c15b55f95ff1 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:38.826663 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 443/877] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:38.831507 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc319215-bad8-49e3-bbbf-ccd0c795ebb7 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:38.831787 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc319215-bad8-49e3-bbbf-ccd0c795ebb7 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:38.832063 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc319215-bad8-49e3-bbbf-ccd0c795ebb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:38.849382 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-fc319215-bad8-49e3-bbbf-ccd0c795ebb7 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:38.850009 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31530e7c-c642-4e00-a1f9-93fecd8f7e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:38.850577 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-fc319215-bad8-49e3-bbbf-ccd0c795ebb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:38.850710 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31530e7c-c642-4e00-a1f9-93fecd8f7e13 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:38.850952 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31530e7c-c642-4e00-a1f9-93fecd8f7e13 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:38.851253 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31530e7c-c642-4e00-a1f9-93fecd8f7e13 None admin] PATH_INFO: `/v3/services` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:38.852032 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31530e7c-c642-4e00-a1f9-93fecd8f7e13 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:38.852340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31530e7c-c642-4e00-a1f9-93fecd8f7e13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:38.854647 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31530e7c-c642-4e00-a1f9-93fecd8f7e13 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:38.863295 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 435/878] 10.222.0.26 () {56 vars in 1122 bytes} [Thu Jun 23 01:44:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:38.866721 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31530e7c-c642-4e00-a1f9-93fecd8f7e13 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:38.920123 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 444/879] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:38 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:44:38.943512 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc319215-bad8-49e3-bbbf-ccd0c795ebb7 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:38.974646 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:38.981879 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31530e7c-c642-4e00-a1f9-93fecd8f7e13 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:39.018609 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250bec38-be26-41ce-aa5f-92dd1364d9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:39.019390 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250bec38-be26-41ce-aa5f-92dd1364d9b6 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:39.019771 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250bec38-be26-41ce-aa5f-92dd1364d9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:39.020136 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250bec38-be26-41ce-aa5f-92dd1364d9b6 None admin] PATH_INFO: `/v3/services` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:39.020952 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250bec38-be26-41ce-aa5f-92dd1364d9b6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:39.021349 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250bec38-be26-41ce-aa5f-92dd1364d9b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:39.022839 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250bec38-be26-41ce-aa5f-92dd1364d9b6 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:39.028083 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 436/880] 10.222.0.26 () {56 vars in 1122 bytes} [Thu Jun 23 01:44:38 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:39.032761 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:39.067242 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc319215-bad8-49e3-bbbf-ccd0c795ebb7 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:39.102741 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677a16c-5ce4-427b-8112-ff65cbdd5171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:39.103461 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677a16c-5ce4-427b-8112-ff65cbdd5171 None admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:39.103738 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677a16c-5ce4-427b-8112-ff65cbdd5171 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:39.104006 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677a16c-5ce4-427b-8112-ff65cbdd5171 None admin] PATH_INFO: `/v3/services` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:39.104767 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d677a16c-5ce4-427b-8112-ff65cbdd5171 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:39.106195 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d677a16c-5ce4-427b-8112-ff65cbdd5171 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:39.123945 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 445/881] 10.222.0.26 () {60 vars in 1141 bytes} [Thu Jun 23 01:44:38 2022] POST /identity/v3/services => generated 269 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:44:39.588232 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eff030-8fe3-4eb7-83e1-f96dd01b17ce None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:39.588232 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eff030-8fe3-4eb7-83e1-f96dd01b17ce None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:39.588232 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eff030-8fe3-4eb7-83e1-f96dd01b17ce None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:39.589128 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 437/882] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:39.594491 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b66ea3-ed32-45eb-ac9d-d9b3b223389c None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:39.595322 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b66ea3-ed32-45eb-ac9d-d9b3b223389c None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:39.595817 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b66ea3-ed32-45eb-ac9d-d9b3b223389c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:39.612997 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-96b66ea3-ed32-45eb-ac9d-d9b3b223389c None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:39.614510 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-96b66ea3-ed32-45eb-ac9d-d9b3b223389c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:39.688381 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 446/883] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:39 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:44:39.712219 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eff030-8fe3-4eb7-83e1-f96dd01b17ce None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:39.822653 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eff030-8fe3-4eb7-83e1-f96dd01b17ce None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:39.857083 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6913fc6-c900-4755-a140-767c422221fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:39.857822 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6913fc6-c900-4755-a140-767c422221fd None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:39.858102 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6913fc6-c900-4755-a140-767c422221fd None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:39.858366 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6913fc6-c900-4755-a140-767c422221fd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:39.869012 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6913fc6-c900-4755-a140-767c422221fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:44:39.877181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-c6913fc6-c900-4755-a140-767c422221fd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 01:44:39.877181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:39.877181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:39.877181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:39.877181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:39.877181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:39.877181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:39.877181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:39.877181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:39.877181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:39.877181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:39.877181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:39.877181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:44:39.877181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:44:39.877181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:44:39.877181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:44:39.878105 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:44:39.878105 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:44:39.878105 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:44:39.878105 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:44:39.878105 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:39.878105 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:39.878105 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:39.878105 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:39.878105 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:39.878105 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:39.878105 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:39.878105 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:39.878105 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:39.878105 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:39.878105 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:39.878105 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:39.878936 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:39.878936 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:39.878936 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:39.878936 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:39.878936 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:44:39.878936 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:44:39.878936 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:44:39.878936 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:44:39.878936 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:44:39.878936 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:44:39.878936 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 01:44:39.878936 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:39.878936 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 438/884] 10.222.0.26 () {56 vars in 1120 bytes} [Thu Jun 23 01:44:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:39.886220 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b66ea3-ed32-45eb-ac9d-d9b3b223389c None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:39.952883 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:39.992348 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b66ea3-ed32-45eb-ac9d-d9b3b223389c None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:40.027250 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c46cea-1fbf-447d-bc10-394caa0c2906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:40.028280 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c46cea-1fbf-447d-bc10-394caa0c2906 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:40.028763 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c46cea-1fbf-447d-bc10-394caa0c2906 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:40.029262 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c46cea-1fbf-447d-bc10-394caa0c2906 None admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:40.030231 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c46cea-1fbf-447d-bc10-394caa0c2906 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:40.030734 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c46cea-1fbf-447d-bc10-394caa0c2906 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:40.032348 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c46cea-1fbf-447d-bc10-394caa0c2906 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:40.041143 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 447/885] 10.222.0.26 () {56 vars in 1115 bytes} [Thu Jun 23 01:44:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:40.047038 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6913fc6-c900-4755-a140-767c422221fd None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:40.071243 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:40.156962 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6913fc6-c900-4755-a140-767c422221fd None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:40.194440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5bba95-1286-4494-99d5-94b9b426a200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:40.195273 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5bba95-1286-4494-99d5-94b9b426a200 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:40.195557 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5bba95-1286-4494-99d5-94b9b426a200 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:40.195825 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5bba95-1286-4494-99d5-94b9b426a200 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:40.206898 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5bba95-1286-4494-99d5-94b9b426a200 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:40.208372 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5bba95-1286-4494-99d5-94b9b426a200 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:40.225877 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-2b5bba95-1286-4494-99d5-94b9b426a200 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:40.225877 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:40.225877 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:40.225877 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:40.225877 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:40.225877 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:40.225877 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:40.225877 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:40.225877 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:40.225877 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:40.225877 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:40.225877 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:40.225877 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:40.225877 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:40.225877 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:40.226821 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:40.226821 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:40.226821 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:40.226821 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:40.226821 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:40.226821 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:40.226821 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:40.226821 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:40.226821 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:40.226821 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:40.226821 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:40.226821 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:40.226821 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:40.226821 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:40.226821 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:40.226821 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:40.226821 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:40.227767 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:40.227767 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:40.227767 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:40.227767 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:40.227767 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 439/886] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:40.234241 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c46cea-1fbf-447d-bc10-394caa0c2906 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:40.341510 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c46cea-1fbf-447d-bc10-394caa0c2906 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:40.367891 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:40.377572 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414f855-5d6e-47d1-a05d-dee1880396fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:40.378517 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414f855-5d6e-47d1-a05d-dee1880396fc None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:40.379026 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414f855-5d6e-47d1-a05d-dee1880396fc None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:40.379547 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414f855-5d6e-47d1-a05d-dee1880396fc None admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:40.380497 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c414f855-5d6e-47d1-a05d-dee1880396fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:40.381155 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c414f855-5d6e-47d1-a05d-dee1880396fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:40.382746 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c414f855-5d6e-47d1-a05d-dee1880396fc None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:40.395832 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 448/887] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:40.401625 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5bba95-1286-4494-99d5-94b9b426a200 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:40.471989 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:40.504181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5bba95-1286-4494-99d5-94b9b426a200 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:40.539238 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3fd882-076a-4a59-b5d4-cf9fafe5a767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:40.540073 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3fd882-076a-4a59-b5d4-cf9fafe5a767 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:40.540351 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3fd882-076a-4a59-b5d4-cf9fafe5a767 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:40.540623 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3fd882-076a-4a59-b5d4-cf9fafe5a767 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:40.551660 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3fd882-076a-4a59-b5d4-cf9fafe5a767 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:40.553195 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3fd882-076a-4a59-b5d4-cf9fafe5a767 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:40.568655 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-cd3fd882-076a-4a59-b5d4-cf9fafe5a767 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 01:44:40.568655 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:40.568655 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:40.568655 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:40.568655 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:40.568655 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:40.568655 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:40.568655 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:40.568655 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:40.568655 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:40.568655 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:40.568655 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:40.568655 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 01:44:40.568655 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 01:44:40.568655 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 01:44:40.568655 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 01:44:40.569659 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:40.569659 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:40.569659 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 01:44:40.569659 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:40.569659 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 01:44:40.569659 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:40.569659 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:40.569659 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:40.569659 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:40.569659 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:40.569659 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:40.569659 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:40.569659 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:40.569659 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:40.569659 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:40.569659 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:40.569659 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:40.569659 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:40.570635 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:40.570635 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:40.570635 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 01:44:40.570635 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 01:44:40.570635 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 01:44:40.570635 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 01:44:40.570635 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 01:44:40.570635 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 01:44:40.570635 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 01:44:40.570635 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:40.570635 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 440/888] 10.222.0.26 () {56 vars in 1117 bytes} [Thu Jun 23 01:44:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:40.577623 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414f855-5d6e-47d1-a05d-dee1880396fc None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:40.680871 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414f855-5d6e-47d1-a05d-dee1880396fc None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:40.716219 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75891e-bcbe-421e-8497-ddbccdb40072 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:40.716742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75891e-bcbe-421e-8497-ddbccdb40072 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:40.716819 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75891e-bcbe-421e-8497-ddbccdb40072 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:40.718603 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 441/889] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:40.719618 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3578010b-fe8d-4c8b-a9cc-8ce1d4c8206a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:40.720511 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3578010b-fe8d-4c8b-a9cc-8ce1d4c8206a None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:40.721049 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3578010b-fe8d-4c8b-a9cc-8ce1d4c8206a None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:40.721521 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3578010b-fe8d-4c8b-a9cc-8ce1d4c8206a None admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:40.722753 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3578010b-fe8d-4c8b-a9cc-8ce1d4c8206a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:40.723287 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3578010b-fe8d-4c8b-a9cc-8ce1d4c8206a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:40.723573 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71d19f-441c-48c7-83b5-60f8d842c050 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:40.723918 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71d19f-441c-48c7-83b5-60f8d842c050 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:40.724223 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71d19f-441c-48c7-83b5-60f8d842c050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:40.726986 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3578010b-fe8d-4c8b-a9cc-8ce1d4c8206a None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:40.741784 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-7d71d19f-441c-48c7-83b5-60f8d842c050 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:40.742852 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-7d71d19f-441c-48c7-83b5-60f8d842c050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:40.755327 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 449/890] 10.222.0.26 () {56 vars in 1149 bytes} [Thu Jun 23 01:44:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:40.757163 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3578010b-fe8d-4c8b-a9cc-8ce1d4c8206a None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:40.812837 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 442/891] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:40 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 01:44:40.813150 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:40.835783 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d71d19f-441c-48c7-83b5-60f8d842c050 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:40.863043 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3578010b-fe8d-4c8b-a9cc-8ce1d4c8206a None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:40.896144 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:40.901914 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695ddec4-351e-4892-adb2-b5c327a26b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:40.902844 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ddec4-351e-4892-adb2-b5c327a26b2b None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:40.903404 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ddec4-351e-4892-adb2-b5c327a26b2b None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:40.903878 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ddec4-351e-4892-adb2-b5c327a26b2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:40.938133 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d71d19f-441c-48c7-83b5-60f8d842c050 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:40.973638 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62218f78-d2a1-45e9-a046-90b4f5fdf46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:40.974257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62218f78-d2a1-45e9-a046-90b4f5fdf46f None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:40.974542 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62218f78-d2a1-45e9-a046-90b4f5fdf46f None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:40.974817 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62218f78-d2a1-45e9-a046-90b4f5fdf46f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:40.975553 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62218f78-d2a1-45e9-a046-90b4f5fdf46f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:40.976882 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62218f78-d2a1-45e9-a046-90b4f5fdf46f None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:40.982737 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695ddec4-351e-4892-adb2-b5c327a26b2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:40.984655 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695ddec4-351e-4892-adb2-b5c327a26b2b None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:40.989489 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-62218f78-d2a1-45e9-a046-90b4f5fdf46f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 01:44:40.989489 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:40.989489 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 01:44:40.989489 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 01:44:40.989489 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 01:44:40.989489 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 01:44:40.989489 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 01:44:40.989489 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:44:40.989489 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 01:44:40.989489 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:40.989489 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 01:44:40.989489 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:40.989489 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:40.989489 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:40.989489 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:40.989489 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:40.989489 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:40.990473 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:40.990473 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:40.990473 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:40.990473 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:40.990473 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:40.990473 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:40.990473 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 01:44:40.990473 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 01:44:40.990473 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 01:44:40.990473 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 01:44:40.990473 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:40.990473 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:40.990473 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:40.990473 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:40.990473 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:40.990473 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:40.991765 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:40.991765 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:40.991765 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:40.991765 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:40.991765 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:40.991765 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:40.991765 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:40.991765 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:40.991765 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:40.991765 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:40.991765 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 01:44:40.991765 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:44:40.991765 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 01:44:40.991765 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:40.995517 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 443/892] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Jun 23 01:44:40 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:44:40.996838 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62218f78-d2a1-45e9-a046-90b4f5fdf46f None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:41.047289 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:41.106748 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62218f78-d2a1-45e9-a046-90b4f5fdf46f None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:41.130121 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 450/893] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:44:40 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 01:44:41.131759 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695ddec4-351e-4892-adb2-b5c327a26b2b None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:41.134026 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:41.145887 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c237172-38d0-479d-8369-9fcd15f9e50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:41.146565 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c237172-38d0-479d-8369-9fcd15f9e50f None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:41.146849 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c237172-38d0-479d-8369-9fcd15f9e50f None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:41.147127 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c237172-38d0-479d-8369-9fcd15f9e50f None admin] PATH_INFO: `/v3/services` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:41.148047 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c237172-38d0-479d-8369-9fcd15f9e50f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:41.148364 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c237172-38d0-479d-8369-9fcd15f9e50f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:41.149754 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c237172-38d0-479d-8369-9fcd15f9e50f None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:41.154628 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 444/894] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Jun 23 01:44:40 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:44:41.155909 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c59163-9b8b-42a7-8cc9-2285359c791b None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:41.156195 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c59163-9b8b-42a7-8cc9-2285359c791b None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:41.156472 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c59163-9b8b-42a7-8cc9-2285359c791b None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:41.159250 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 445/895] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:41.160996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c59163-9b8b-42a7-8cc9-2285359c791b None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:41.236606 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695ddec4-351e-4892-adb2-b5c327a26b2b None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:41.261672 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c59163-9b8b-42a7-8cc9-2285359c791b None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:41.270103 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb52c1-9c53-45b2-91f0-1fd93a49e39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:41.270887 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb52c1-9c53-45b2-91f0-1fd93a49e39f None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:41.271161 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb52c1-9c53-45b2-91f0-1fd93a49e39f None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:41.271493 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb52c1-9c53-45b2-91f0-1fd93a49e39f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:41.281618 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eb52c1-9c53-45b2-91f0-1fd93a49e39f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:41.283067 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eb52c1-9c53-45b2-91f0-1fd93a49e39f None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:41.285918 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:41.302077 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-d9eb52c1-9c53-45b2-91f0-1fd93a49e39f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:41.302077 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:41.302077 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:41.302077 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:41.302077 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:41.302077 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:41.302077 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:41.302077 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:41.302077 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:41.302077 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:41.302077 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:41.302077 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:41.302077 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:41.302077 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:41.302077 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:41.303088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:41.303088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:41.303088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:41.303088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:41.303088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:41.303088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:41.303088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:41.303088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:41.303088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:41.303088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:41.303088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:41.303088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:41.303088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:41.303088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:41.303088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:41.303088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:41.303088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:41.303992 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:41.303992 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:41.303992 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:41.303992 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:41.304209 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03859722-77c3-4d61-ab96-d2af7e4f9118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:41.304209 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03859722-77c3-4d61-ab96-d2af7e4f9118 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:41.304209 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03859722-77c3-4d61-ab96-d2af7e4f9118 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:41.304413 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 451/896] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:41.304798 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03859722-77c3-4d61-ab96-d2af7e4f9118 None admin] PATH_INFO: `/v3/services` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:41.304798 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03859722-77c3-4d61-ab96-d2af7e4f9118 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:41.304798 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03859722-77c3-4d61-ab96-d2af7e4f9118 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:41.304972 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4020aa2e-2687-4e53-ac2c-bae9e1506ce5 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:41.305120 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4020aa2e-2687-4e53-ac2c-bae9e1506ce5 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:41.305601 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4020aa2e-2687-4e53-ac2c-bae9e1506ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:41.306104 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03859722-77c3-4d61-ab96-d2af7e4f9118 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:41.315334 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 446/897] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Jun 23 01:44:41 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:44:41.315334 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837d5d7-8aa3-40c0-8e84-c5c34e61c86d None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:41.315334 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837d5d7-8aa3-40c0-8e84-c5c34e61c86d None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:41.315334 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837d5d7-8aa3-40c0-8e84-c5c34e61c86d None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:41.319805 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 447/898] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:41.319805 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837d5d7-8aa3-40c0-8e84-c5c34e61c86d None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:41.323416 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-4020aa2e-2687-4e53-ac2c-bae9e1506ce5 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:41.324614 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-4020aa2e-2687-4e53-ac2c-bae9e1506ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:41.369607 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:41.391520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 452/899] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:41 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:44:41.392358 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4020aa2e-2687-4e53-ac2c-bae9e1506ce5 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:41.420414 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837d5d7-8aa3-40c0-8e84-c5c34e61c86d None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:41.455878 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5895a282-95bc-4d48-9486-9f85241ec32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:41.456552 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5895a282-95bc-4d48-9486-9f85241ec32d None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:41.456837 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5895a282-95bc-4d48-9486-9f85241ec32d None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:41.457118 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5895a282-95bc-4d48-9486-9f85241ec32d None admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:41.457848 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5895a282-95bc-4d48-9486-9f85241ec32d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:41.458157 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5895a282-95bc-4d48-9486-9f85241ec32d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:41.459557 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5895a282-95bc-4d48-9486-9f85241ec32d None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:41.471859 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 448/900] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:41.473031 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce3036-eef5-48d5-91bc-86d7da64eba0 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:41.473320 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce3036-eef5-48d5-91bc-86d7da64eba0 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:41.473595 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce3036-eef5-48d5-91bc-86d7da64eba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:41.490025 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-e9ce3036-eef5-48d5-91bc-86d7da64eba0 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:41.491158 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-e9ce3036-eef5-48d5-91bc-86d7da64eba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:41.495984 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4020aa2e-2687-4e53-ac2c-bae9e1506ce5 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:41.521054 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:41.529918 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e5ad36-d4c5-4296-bdfd-47af31e76219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:41.530896 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5ad36-d4c5-4296-bdfd-47af31e76219 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:41.530896 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5ad36-d4c5-4296-bdfd-47af31e76219 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:41.531105 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5ad36-d4c5-4296-bdfd-47af31e76219 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:41.531925 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e5ad36-d4c5-4296-bdfd-47af31e76219 None admin] RBAC: Adding query filter params (interface=public, service_id=6e6fcd5ea7014578a24b60c51574e078, region_id=RegionOne) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:41.532231 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e5ad36-d4c5-4296-bdfd-47af31e76219 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:41.533607 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e5ad36-d4c5-4296-bdfd-47af31e76219 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:41.540860 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 453/901] 10.222.0.26 () {56 vars in 1251 bytes} [Thu Jun 23 01:44:41 2022] GET /identity/v3/endpoints?service_id=6e6fcd5ea7014578a24b60c51574e078&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:41.541599 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e5ad36-d4c5-4296-bdfd-47af31e76219 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:41.558728 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 449/902] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:41 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:44:41.559591 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ce3036-eef5-48d5-91bc-86d7da64eba0 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:41.602428 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:41.642617 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e5ad36-d4c5-4296-bdfd-47af31e76219 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:41.657662 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ce3036-eef5-48d5-91bc-86d7da64eba0 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:41.675400 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94cd73a-56a3-4b73-88ae-8b283c193caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:41.676052 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94cd73a-56a3-4b73-88ae-8b283c193caa None admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:41.676329 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94cd73a-56a3-4b73-88ae-8b283c193caa None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:41.676599 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94cd73a-56a3-4b73-88ae-8b283c193caa None admin] PATH_INFO: `/v3/services` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:41.677338 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94cd73a-56a3-4b73-88ae-8b283c193caa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:41.678684 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94cd73a-56a3-4b73-88ae-8b283c193caa None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:41.691721 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79511522-b02b-4d66-a8c9-a20da056a470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:41.692517 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79511522-b02b-4d66-a8c9-a20da056a470 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:41.692793 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79511522-b02b-4d66-a8c9-a20da056a470 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:41.693062 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79511522-b02b-4d66-a8c9-a20da056a470 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:41.693337 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 454/903] 10.222.0.26 () {60 vars in 1141 bytes} [Thu Jun 23 01:44:41 2022] POST /identity/v3/services => generated 241 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:44:41.694161 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94cd73a-56a3-4b73-88ae-8b283c193caa None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:41.700703 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79511522-b02b-4d66-a8c9-a20da056a470 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:41.702231 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79511522-b02b-4d66-a8c9-a20da056a470 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:41.718018 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-79511522-b02b-4d66-a8c9-a20da056a470 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:41.718018 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:41.718018 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:41.718018 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:41.718018 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:41.718018 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:41.718018 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:41.718018 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:41.718018 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:41.718018 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:41.718018 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:41.718018 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:41.718018 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 01:44:41.718018 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 01:44:41.718018 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 01:44:41.719482 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 01:44:41.719482 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:41.719482 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:41.719482 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:41.719482 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:41.719482 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:41.719482 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:41.719482 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:41.719482 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:41.719482 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:41.719482 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:41.719482 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:41.719482 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:41.719482 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:41.719482 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:41.719482 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:41.719482 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:41.720600 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 01:44:41.720600 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 01:44:41.720600 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 01:44:41.720600 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 01:44:41.720600 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 01:44:41.720600 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 01:44:41.720600 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:41.720600 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:41.720600 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 450/904] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:44:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:41.723672 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79511522-b02b-4d66-a8c9-a20da056a470 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:41.758390 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:41.824065 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94cd73a-56a3-4b73-88ae-8b283c193caa None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:41.848922 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79511522-b02b-4d66-a8c9-a20da056a470 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:41.850528 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:41.864038 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db69dcd-4b27-4913-90e4-0555f63bb04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:41.864711 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db69dcd-4b27-4913-90e4-0555f63bb04b None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:41.864992 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db69dcd-4b27-4913-90e4-0555f63bb04b None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:41.865268 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db69dcd-4b27-4913-90e4-0555f63bb04b None admin] PATH_INFO: `/v3/services` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:41.865965 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db69dcd-4b27-4913-90e4-0555f63bb04b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:41.867442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db69dcd-4b27-4913-90e4-0555f63bb04b None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:41.874786 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 455/905] 10.222.0.26 () {56 vars in 1087 bytes} [Thu Jun 23 01:44:41 2022] GET /identity/v3/services => generated 1497 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:44:41.875655 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db69dcd-4b27-4913-90e4-0555f63bb04b None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:41.903252 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d935bd-58d3-4003-bdcf-72069694bec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:41.903252 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d935bd-58d3-4003-bdcf-72069694bec0 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:41.903252 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d935bd-58d3-4003-bdcf-72069694bec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:41.903252 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d935bd-58d3-4003-bdcf-72069694bec0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:41.903706 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d935bd-58d3-4003-bdcf-72069694bec0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:41.904697 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d935bd-58d3-4003-bdcf-72069694bec0 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:41.922654 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-61d935bd-58d3-4003-bdcf-72069694bec0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 01:44:41.922654 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:41.922654 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 01:44:41.922654 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 01:44:41.922654 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 01:44:41.922654 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 01:44:41.922654 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 01:44:41.922654 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:44:41.922654 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 01:44:41.922654 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:41.922654 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 01:44:41.922654 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:41.922654 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:41.922654 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:41.922654 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:41.922654 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:41.923893 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:41.923893 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:41.923893 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:41.923893 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:41.923893 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:41.923893 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:41.923893 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:41.923893 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 01:44:41.923893 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 01:44:41.923893 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 01:44:41.923893 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 01:44:41.923893 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:41.923893 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:41.923893 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:41.923893 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:41.923893 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:41.924971 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:41.924971 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:41.924971 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:41.924971 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:41.924971 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:41.924971 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:41.924971 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:41.924971 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:41.924971 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:41.924971 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:41.924971 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:41.924971 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 01:44:41.924971 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:44:41.924971 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 01:44:41.924971 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:41.924971 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 451/906] 10.222.0.26 () {56 vars in 1132 bytes} [Thu Jun 23 01:44:41 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:41.930893 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d935bd-58d3-4003-bdcf-72069694bec0 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:42.003997 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db69dcd-4b27-4913-90e4-0555f63bb04b None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:42.017185 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:42.049864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23764bb-2c0d-4d47-b6d0-745b223aab9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:42.050530 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23764bb-2c0d-4d47-b6d0-745b223aab9e None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:42.050808 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23764bb-2c0d-4d47-b6d0-745b223aab9e None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:42.051084 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23764bb-2c0d-4d47-b6d0-745b223aab9e None admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:42.052344 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23764bb-2c0d-4d47-b6d0-745b223aab9e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:42.052669 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23764bb-2c0d-4d47-b6d0-745b223aab9e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:42.054268 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23764bb-2c0d-4d47-b6d0-745b223aab9e None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:42.066332 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d935bd-58d3-4003-bdcf-72069694bec0 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:42.071250 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 456/907] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jun 23 01:44:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:42.076232 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23764bb-2c0d-4d47-b6d0-745b223aab9e None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:42.113097 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce17b179-c5bf-41be-a23b-8ac04234ab00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:42.113820 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce17b179-c5bf-41be-a23b-8ac04234ab00 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:42.114104 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce17b179-c5bf-41be-a23b-8ac04234ab00 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:42.114384 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce17b179-c5bf-41be-a23b-8ac04234ab00 None admin] PATH_INFO: `/v3/services` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:42.115103 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce17b179-c5bf-41be-a23b-8ac04234ab00 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:42.115467 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce17b179-c5bf-41be-a23b-8ac04234ab00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:42.116911 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce17b179-c5bf-41be-a23b-8ac04234ab00 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:42.122215 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 452/908] 10.222.0.26 () {56 vars in 1126 bytes} [Thu Jun 23 01:44:41 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:42.127480 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce17b179-c5bf-41be-a23b-8ac04234ab00 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:42.158782 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:42.200544 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23764bb-2c0d-4d47-b6d0-745b223aab9e None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:42.238076 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce17b179-c5bf-41be-a23b-8ac04234ab00 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:42.239022 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16212fc0-7b8b-4856-aa5d-dddea8f76c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:42.239788 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16212fc0-7b8b-4856-aa5d-dddea8f76c35 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:42.240067 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16212fc0-7b8b-4856-aa5d-dddea8f76c35 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:42.240338 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16212fc0-7b8b-4856-aa5d-dddea8f76c35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:42.241450 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16212fc0-7b8b-4856-aa5d-dddea8f76c35 None admin] RBAC: Adding query filter params (role.id=f173bf8d88d34d82881f578a6c10a7ce, scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c, user.id=168e6181ad22460990f1f7361b483364) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:42.241761 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16212fc0-7b8b-4856-aa5d-dddea8f76c35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:42.243222 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16212fc0-7b8b-4856-aa5d-dddea8f76c35 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:42.255519 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 457/909] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:44:42 2022] GET /identity/v3/role_assignments?user.id=168e6181ad22460990f1f7361b483364&scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c&role.id=f173bf8d88d34d82881f578a6c10a7ce => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:44:42.281968 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34217c7a-6960-4d8a-bfb1-f465bc62631d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:42.282568 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34217c7a-6960-4d8a-bfb1-f465bc62631d None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:42.282852 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34217c7a-6960-4d8a-bfb1-f465bc62631d None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:42.283130 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34217c7a-6960-4d8a-bfb1-f465bc62631d None admin] PATH_INFO: `/v3/services` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:42.283950 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34217c7a-6960-4d8a-bfb1-f465bc62631d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:42.284269 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34217c7a-6960-4d8a-bfb1-f465bc62631d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:42.285694 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34217c7a-6960-4d8a-bfb1-f465bc62631d None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:42.295027 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 453/910] 10.222.0.26 () {56 vars in 1126 bytes} [Thu Jun 23 01:44:42 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:42.304402 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16212fc0-7b8b-4856-aa5d-dddea8f76c35 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:42.342973 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:42.415273 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16212fc0-7b8b-4856-aa5d-dddea8f76c35 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:42.456505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b3b2a-addd-4021-b47a-f7dec8ee4104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:42.457187 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b3b2a-addd-4021-b47a-f7dec8ee4104 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:42.457462 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b3b2a-addd-4021-b47a-f7dec8ee4104 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:42.457731 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b3b2a-addd-4021-b47a-f7dec8ee4104 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:42.458536 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966b3b2a-addd-4021-b47a-f7dec8ee4104 None admin] RBAC: Adding query filter params (interface=public, service_id=5eead55df1ed4911a0e9f28b3897367a, region_id=RegionOne) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:42.458838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966b3b2a-addd-4021-b47a-f7dec8ee4104 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:42.460245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966b3b2a-addd-4021-b47a-f7dec8ee4104 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:42.465967 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 458/911] 10.222.0.26 () {56 vars in 1251 bytes} [Thu Jun 23 01:44:42 2022] GET /identity/v3/endpoints?service_id=5eead55df1ed4911a0e9f28b3897367a&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:42.471452 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34217c7a-6960-4d8a-bfb1-f465bc62631d None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:42.574180 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:42.584436 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34217c7a-6960-4d8a-bfb1-f465bc62631d None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:42.618226 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec3b1a-3ce1-4963-9c5a-18cec38f6ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:42.618902 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec3b1a-3ce1-4963-9c5a-18cec38f6ff5 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:42.619187 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec3b1a-3ce1-4963-9c5a-18cec38f6ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:42.619497 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec3b1a-3ce1-4963-9c5a-18cec38f6ff5 None admin] PATH_INFO: `/v3/services` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:42.620191 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ec3b1a-3ce1-4963-9c5a-18cec38f6ff5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:42.621573 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ec3b1a-3ce1-4963-9c5a-18cec38f6ff5 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:42.628528 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 454/912] 10.222.0.26 () {56 vars in 1087 bytes} [Thu Jun 23 01:44:42 2022] GET /identity/v3/services => generated 1497 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:44:43.809170 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3448f6b-d474-4b47-a3b2-8aaed36dcef7 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:43.809804 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3448f6b-d474-4b47-a3b2-8aaed36dcef7 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:43.809804 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3448f6b-d474-4b47-a3b2-8aaed36dcef7 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:43.811349 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 459/913] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:43.816755 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33824fa1-7107-4828-81a5-26dc8ba135a1 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:43.817174 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33824fa1-7107-4828-81a5-26dc8ba135a1 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:43.817351 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33824fa1-7107-4828-81a5-26dc8ba135a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:43.834808 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-33824fa1-7107-4828-81a5-26dc8ba135a1 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:43.836034 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-33824fa1-7107-4828-81a5-26dc8ba135a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:43.906222 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 455/914] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:44:43.928996 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3448f6b-d474-4b47-a3b2-8aaed36dcef7 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:43.976313 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364956b4-4abe-4fff-a142-37a2024fce00 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:43.976661 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364956b4-4abe-4fff-a142-37a2024fce00 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:43.976869 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364956b4-4abe-4fff-a142-37a2024fce00 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:43.978510 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 456/915] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 01:44:43.983408 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11452ab7-8acf-4f26-a9ea-4158e2b28630 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:43.983683 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11452ab7-8acf-4f26-a9ea-4158e2b28630 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:43.983960 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11452ab7-8acf-4f26-a9ea-4158e2b28630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:44.001285 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-11452ab7-8acf-4f26-a9ea-4158e2b28630 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:44.002958 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-11452ab7-8acf-4f26-a9ea-4158e2b28630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:44.043422 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:44.050256 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3448f6b-d474-4b47-a3b2-8aaed36dcef7 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:44.080946 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 457/916] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 01:44:44.091146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a225cf-012c-4050-adcb-ce496094ab02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:44.092266 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a225cf-012c-4050-adcb-ce496094ab02 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:44.092771 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a225cf-012c-4050-adcb-ce496094ab02 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:44.093226 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a225cf-012c-4050-adcb-ce496094ab02 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:44.094125 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a225cf-012c-4050-adcb-ce496094ab02 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:44.095866 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a225cf-012c-4050-adcb-ce496094ab02 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:44.098436 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11452ab7-8acf-4f26-a9ea-4158e2b28630 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:44.113694 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-86a225cf-012c-4050-adcb-ce496094ab02 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 01:44:44.113694 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:44.113694 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 01:44:44.113694 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 01:44:44.113694 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 01:44:44.113694 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 01:44:44.113694 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 01:44:44.113694 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:44:44.113694 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 01:44:44.113694 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:44.113694 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 01:44:44.113694 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:44.113694 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:44.113694 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:44.113694 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:44.113694 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:44.113694 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:44.114826 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:44.114826 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:44.114826 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:44.114826 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:44.114826 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:44.114826 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:44.114826 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 01:44:44.114826 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 01:44:44.114826 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 01:44:44.114826 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 01:44:44.114826 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:44.114826 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:44.114826 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:44.114826 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:44.114826 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:44.114826 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:44.115765 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:44.115765 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:44.115765 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:44.115765 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:44.115765 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:44.115765 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:44.115765 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:44.115765 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:44.115765 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:44.115765 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:44.115765 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 01:44:44.115765 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:44:44.115765 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 01:44:44.115765 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:44.115765 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 460/917] 10.222.0.26 () {56 vars in 1126 bytes} [Thu Jun 23 01:44:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:44.123020 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a225cf-012c-4050-adcb-ce496094ab02 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:44.127621 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:44.204497 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11452ab7-8acf-4f26-a9ea-4158e2b28630 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:44.243546 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b9dac-9c9c-4033-beea-7da051c84bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:44.244764 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b9dac-9c9c-4033-beea-7da051c84bfb None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:44.245250 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b9dac-9c9c-4033-beea-7da051c84bfb None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:44.245714 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b9dac-9c9c-4033-beea-7da051c84bfb None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:44.246658 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01b9dac-9c9c-4033-beea-7da051c84bfb None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:44.248297 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01b9dac-9c9c-4033-beea-7da051c84bfb None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:44.249688 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a225cf-012c-4050-adcb-ce496094ab02 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:44.267031 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-d01b9dac-9c9c-4033-beea-7da051c84bfb None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 01:44:44.267031 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:44.267031 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 01:44:44.267031 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 01:44:44.267031 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 01:44:44.267031 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 01:44:44.267031 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 01:44:44.267031 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:44:44.267031 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 01:44:44.267031 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:44.267031 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 01:44:44.267031 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:44.267031 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:44.267031 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:44.267031 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:44.267031 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:44.267031 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:44.268181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:44.268181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:44.268181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:44.268181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:44.268181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:44.268181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:44.268181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 01:44:44.268181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 01:44:44.268181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 01:44:44.268181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 01:44:44.268181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:44.268181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:44.268181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:44.268181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:44.268181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:44.268181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:44.269013 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:44.269013 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:44.269013 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:44.269013 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:44.269013 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:44.269013 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:44.269013 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:44.269013 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:44.269013 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:44.269013 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:44.269013 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 01:44:44.269013 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:44:44.269013 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 01:44:44.269013 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:44.272094 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 458/918] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Jun 23 01:44:44 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:44:44.278158 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b9dac-9c9c-4033-beea-7da051c84bfb None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:44.293472 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15860b44-fc12-4829-8c47-b49b69375cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:44.294197 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15860b44-fc12-4829-8c47-b49b69375cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:44.294968 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15860b44-fc12-4829-8c47-b49b69375cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:44.294968 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15860b44-fc12-4829-8c47-b49b69375cd8 None admin] PATH_INFO: `/v3/services` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:44.295539 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15860b44-fc12-4829-8c47-b49b69375cd8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:44.295903 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15860b44-fc12-4829-8c47-b49b69375cd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:44.297357 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15860b44-fc12-4829-8c47-b49b69375cd8 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:44.307802 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 461/919] 10.222.0.26 () {56 vars in 1122 bytes} [Thu Jun 23 01:44:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:44.308684 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15860b44-fc12-4829-8c47-b49b69375cd8 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:44.342849 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:44.373740 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:44.396981 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b9dac-9c9c-4033-beea-7da051c84bfb None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:44.431395 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15860b44-fc12-4829-8c47-b49b69375cd8 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:44.432649 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8f063e-69a2-4491-ae99-494a35f51b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:44.433774 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f063e-69a2-4491-ae99-494a35f51b88 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:44.434113 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f063e-69a2-4491-ae99-494a35f51b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:44.434456 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f063e-69a2-4491-ae99-494a35f51b88 None admin] PATH_INFO: `/v3/services` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:44.435162 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8f063e-69a2-4491-ae99-494a35f51b88 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:44.435494 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8f063e-69a2-4491-ae99-494a35f51b88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:44.436857 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8f063e-69a2-4491-ae99-494a35f51b88 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:44.446579 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 459/920] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Jun 23 01:44:44 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:44:44.447810 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc994c3-cd19-4e7b-916d-6923515aa7e1 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:44.448059 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc994c3-cd19-4e7b-916d-6923515aa7e1 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:44.448338 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc994c3-cd19-4e7b-916d-6923515aa7e1 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:44.451656 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 460/921] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:44.452928 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc994c3-cd19-4e7b-916d-6923515aa7e1 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:44.473802 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d34b02b-1a95-49f3-a576-290b80a571f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:44.474466 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34b02b-1a95-49f3-a576-290b80a571f6 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:44.474746 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34b02b-1a95-49f3-a576-290b80a571f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:44.475022 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34b02b-1a95-49f3-a576-290b80a571f6 None admin] PATH_INFO: `/v3/services` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:44.475777 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d34b02b-1a95-49f3-a576-290b80a571f6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:44.476078 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d34b02b-1a95-49f3-a576-290b80a571f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:44.477451 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d34b02b-1a95-49f3-a576-290b80a571f6 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:44.487171 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 462/922] 10.222.0.26 () {56 vars in 1122 bytes} [Thu Jun 23 01:44:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:44:44.487171 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169710d9-6c9a-49ce-92f3-6af5bdee2b85 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:44.487171 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169710d9-6c9a-49ce-92f3-6af5bdee2b85 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:44.487171 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169710d9-6c9a-49ce-92f3-6af5bdee2b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:44.501249 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-169710d9-6c9a-49ce-92f3-6af5bdee2b85 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:44.502481 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-169710d9-6c9a-49ce-92f3-6af5bdee2b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:44.572697 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc994c3-cd19-4e7b-916d-6923515aa7e1 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:44.580780 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 463/923] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:44:44.580780 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169710d9-6c9a-49ce-92f3-6af5bdee2b85 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:44.597301 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:44.621723 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b368c5a-57ce-4925-921d-28c16740086e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:44.622587 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b368c5a-57ce-4925-921d-28c16740086e None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:44.622873 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b368c5a-57ce-4925-921d-28c16740086e None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:44.623539 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b368c5a-57ce-4925-921d-28c16740086e None admin] PATH_INFO: `/v3/services` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:44.624270 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b368c5a-57ce-4925-921d-28c16740086e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:44.624582 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b368c5a-57ce-4925-921d-28c16740086e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:44.625966 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b368c5a-57ce-4925-921d-28c16740086e None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:44.631129 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 461/924] 10.222.0.26 () {56 vars in 1114 bytes} [Thu Jun 23 01:44:44 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:44.631950 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b368c5a-57ce-4925-921d-28c16740086e None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:44.644403 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:44.675461 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169710d9-6c9a-49ce-92f3-6af5bdee2b85 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:44.708800 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392b906c-eb01-4407-b4be-23dc01fb2a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:44.709454 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b906c-eb01-4407-b4be-23dc01fb2a31 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:44.709738 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b906c-eb01-4407-b4be-23dc01fb2a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:44.710013 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b906c-eb01-4407-b4be-23dc01fb2a31 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:44.710799 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392b906c-eb01-4407-b4be-23dc01fb2a31 None admin] RBAC: Adding query filter params (interface=public, service_id=52022d5659834a0f8e19e5fded581053, region_id=RegionOne) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:44.711106 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392b906c-eb01-4407-b4be-23dc01fb2a31 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:44.712532 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392b906c-eb01-4407-b4be-23dc01fb2a31 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:44.717612 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 464/925] 10.222.0.26 () {56 vars in 1251 bytes} [Thu Jun 23 01:44:44 2022] GET /identity/v3/endpoints?service_id=52022d5659834a0f8e19e5fded581053&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:44.718624 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb4bd79-3aee-4f91-9424-7717f17e6ae2 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:44.718909 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb4bd79-3aee-4f91-9424-7717f17e6ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:44.719182 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb4bd79-3aee-4f91-9424-7717f17e6ae2 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:44.720415 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 465/926] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:44.721149 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb4bd79-3aee-4f91-9424-7717f17e6ae2 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:44.740468 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b368c5a-57ce-4925-921d-28c16740086e None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:44.773805 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bfb9b3-0c4b-4d25-8974-dc844fb8143a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:44.774628 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bfb9b3-0c4b-4d25-8974-dc844fb8143a None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:44.774913 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bfb9b3-0c4b-4d25-8974-dc844fb8143a None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:44.775191 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bfb9b3-0c4b-4d25-8974-dc844fb8143a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:44.785399 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bfb9b3-0c4b-4d25-8974-dc844fb8143a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:44:44.793574 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-38bfb9b3-0c4b-4d25-8974-dc844fb8143a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 01:44:44.793574 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:44.793574 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:44.793574 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:44.793574 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:44.793574 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:44.793574 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:44.793574 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:44.793574 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:44.793574 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:44.793574 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:44.793574 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:44.793574 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:44:44.793574 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:44:44.793574 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:44:44.793574 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:44:44.794550 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:44:44.794550 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:44:44.794550 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:44:44.794550 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:44:44.794550 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:44.794550 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:44.794550 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:44.794550 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:44.794550 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:44.794550 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:44.794550 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:44.794550 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:44.794550 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:44.794550 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:44.794550 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:44.794550 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:44.795394 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:44.795394 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:44.795394 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:44.795394 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:44.795394 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:44:44.795394 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:44:44.795394 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:44:44.795394 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:44:44.795394 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:44:44.795394 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:44:44.795394 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 01:44:44.795394 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:44.795394 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 462/927] 10.222.0.26 () {56 vars in 1120 bytes} [Thu Jun 23 01:44:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:44.796097 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bfb9b3-0c4b-4d25-8974-dc844fb8143a None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:44.809426 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:44.825893 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb4bd79-3aee-4f91-9424-7717f17e6ae2 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:44.860301 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf276f14-7de1-478c-9614-c347f48114b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:44.860887 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf276f14-7de1-478c-9614-c347f48114b2 None admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:44.861164 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf276f14-7de1-478c-9614-c347f48114b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:44.861455 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf276f14-7de1-478c-9614-c347f48114b2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:44.862193 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf276f14-7de1-478c-9614-c347f48114b2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:44.863600 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf276f14-7de1-478c-9614-c347f48114b2 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:44.864639 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:44.882598 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 466/928] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Jun 23 01:44:44 2022] POST /identity/v3/endpoints => generated 346 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:44:44.883921 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7317249-5644-4055-9df4-96ee28e68d8a None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:44.884206 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7317249-5644-4055-9df4-96ee28e68d8a None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:44.884476 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7317249-5644-4055-9df4-96ee28e68d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:44.902105 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-c7317249-5644-4055-9df4-96ee28e68d8a None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:44.903351 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-c7317249-5644-4055-9df4-96ee28e68d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:44.914441 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bfb9b3-0c4b-4d25-8974-dc844fb8143a None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:44.963385 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ef4c2-6c79-4b5f-bb39-90edc7abd7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:44.964793 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ef4c2-6c79-4b5f-bb39-90edc7abd7de None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:44.965293 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ef4c2-6c79-4b5f-bb39-90edc7abd7de None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:44.965812 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ef4c2-6c79-4b5f-bb39-90edc7abd7de None admin] PATH_INFO: `/v3/services` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:44.966722 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3ef4c2-6c79-4b5f-bb39-90edc7abd7de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:44.968971 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3ef4c2-6c79-4b5f-bb39-90edc7abd7de None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:44.977204 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 463/929] 10.222.0.26 () {56 vars in 1087 bytes} [Thu Jun 23 01:44:44 2022] GET /identity/v3/services => generated 1497 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:44:44.978248 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ef4c2-6c79-4b5f-bb39-90edc7abd7de None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:44.993069 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 467/930] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:44:45.018147 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7317249-5644-4055-9df4-96ee28e68d8a None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:45.113819 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:45.132632 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ef4c2-6c79-4b5f-bb39-90edc7abd7de None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:45.161645 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7317249-5644-4055-9df4-96ee28e68d8a None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:45.166510 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:45.177596 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e7d218-32be-4af5-be72-a6b5652b37eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:45.178281 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e7d218-32be-4af5-be72-a6b5652b37eb None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:45.178561 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e7d218-32be-4af5-be72-a6b5652b37eb None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:45.178836 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e7d218-32be-4af5-be72-a6b5652b37eb None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:45.180279 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e7d218-32be-4af5-be72-a6b5652b37eb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:45.180279 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e7d218-32be-4af5-be72-a6b5652b37eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:45.181616 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e7d218-32be-4af5-be72-a6b5652b37eb None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:45.190966 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 464/931] 10.222.0.26 () {56 vars in 1115 bytes} [Thu Jun 23 01:44:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:45.196640 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e7d218-32be-4af5-be72-a6b5652b37eb None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:45.208502 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f52df-8995-4a14-93d1-f0caf3872116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:45.210034 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f52df-8995-4a14-93d1-f0caf3872116 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:45.210880 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f52df-8995-4a14-93d1-f0caf3872116 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:45.211344 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f52df-8995-4a14-93d1-f0caf3872116 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:45.212450 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f52df-8995-4a14-93d1-f0caf3872116 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:45.214735 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f52df-8995-4a14-93d1-f0caf3872116 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:45.232158 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-3b6f52df-8995-4a14-93d1-f0caf3872116 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 01:44:45.232158 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:45.232158 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 01:44:45.232158 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 01:44:45.232158 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 01:44:45.232158 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 01:44:45.232158 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 01:44:45.232158 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:44:45.232158 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 01:44:45.232158 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:45.232158 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 01:44:45.232158 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:45.232158 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:45.232158 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:45.232158 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:45.232158 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:45.233350 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:45.233350 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:45.233350 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:45.233350 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:45.233350 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:45.233350 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:45.233350 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:45.233350 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 01:44:45.233350 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 01:44:45.233350 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 01:44:45.233350 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 01:44:45.233350 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:45.233350 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:45.233350 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:45.233350 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:45.233350 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:45.234398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:45.234398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:45.234398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:45.234398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:45.234398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:45.234398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:45.234398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:45.234398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:45.234398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:45.234398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:45.234398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:45.234398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 01:44:45.234398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:44:45.234398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 01:44:45.234398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:45.237284 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 468/932] 10.222.0.26 () {56 vars in 1132 bytes} [Thu Jun 23 01:44:45 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:44:45.243156 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f52df-8995-4a14-93d1-f0caf3872116 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:45.321726 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e7d218-32be-4af5-be72-a6b5652b37eb None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:45.357702 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd070b7d-1b6d-4f51-a954-e1c31d73a1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:45.358780 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd070b7d-1b6d-4f51-a954-e1c31d73a1aa None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:45.359195 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd070b7d-1b6d-4f51-a954-e1c31d73a1aa None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:45.359661 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd070b7d-1b6d-4f51-a954-e1c31d73a1aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:45.371931 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd070b7d-1b6d-4f51-a954-e1c31d73a1aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:45.373621 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd070b7d-1b6d-4f51-a954-e1c31d73a1aa None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:45.374752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f52df-8995-4a14-93d1-f0caf3872116 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:45.392157 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-cd070b7d-1b6d-4f51-a954-e1c31d73a1aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:45.392157 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:45.392157 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:45.392157 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:45.392157 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:45.392157 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:45.392157 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:45.392157 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:45.392157 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:45.392157 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:45.392157 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:45.392157 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:45.392157 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:45.392157 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:45.392157 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:45.393225 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:45.393225 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:45.393225 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:45.393225 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:45.393225 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:45.393225 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:45.393225 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:45.393225 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:45.393225 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:45.393225 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:45.393225 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:45.393225 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:45.393225 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:45.393225 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:45.393225 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:45.393225 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:45.393225 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:45.394118 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:45.394118 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:45.394118 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:45.394118 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:45.400905 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 465/933] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:45.402003 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd070b7d-1b6d-4f51-a954-e1c31d73a1aa None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:45.411768 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803e67d4-7bdc-4777-95a0-996bfa6c908a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:45.412868 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e67d4-7bdc-4777-95a0-996bfa6c908a None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:45.412868 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e67d4-7bdc-4777-95a0-996bfa6c908a None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:45.413082 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e67d4-7bdc-4777-95a0-996bfa6c908a None admin] PATH_INFO: `/v3/services` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:45.413862 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803e67d4-7bdc-4777-95a0-996bfa6c908a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:45.414180 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803e67d4-7bdc-4777-95a0-996bfa6c908a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:45.415972 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803e67d4-7bdc-4777-95a0-996bfa6c908a None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:45.417096 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:45.427152 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 469/934] 10.222.0.26 () {56 vars in 1126 bytes} [Thu Jun 23 01:44:45 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:44:45.428696 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803e67d4-7bdc-4777-95a0-996bfa6c908a None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:45.491650 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:45.507453 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd070b7d-1b6d-4f51-a954-e1c31d73a1aa None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:45.535713 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803e67d4-7bdc-4777-95a0-996bfa6c908a None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:45.541976 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab1a3f-981c-49cf-acaa-14f78779769a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:45.542649 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab1a3f-981c-49cf-acaa-14f78779769a None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:45.542928 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab1a3f-981c-49cf-acaa-14f78779769a None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:45.543205 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab1a3f-981c-49cf-acaa-14f78779769a None admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:45.543961 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ab1a3f-981c-49cf-acaa-14f78779769a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:45.544268 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ab1a3f-981c-49cf-acaa-14f78779769a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:45.545751 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ab1a3f-981c-49cf-acaa-14f78779769a None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:45.562471 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 466/935] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:45.563527 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab1a3f-981c-49cf-acaa-14f78779769a None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:45.574003 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29106512-35be-41dc-9a38-abc160b429bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:45.575066 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29106512-35be-41dc-9a38-abc160b429bd None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:45.575579 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29106512-35be-41dc-9a38-abc160b429bd None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:45.576056 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29106512-35be-41dc-9a38-abc160b429bd None admin] PATH_INFO: `/v3/services` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:45.577009 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29106512-35be-41dc-9a38-abc160b429bd None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:45.577522 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29106512-35be-41dc-9a38-abc160b429bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:45.579122 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29106512-35be-41dc-9a38-abc160b429bd None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:45.590004 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 470/936] 10.222.0.26 () {56 vars in 1126 bytes} [Thu Jun 23 01:44:45 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:45.591120 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29106512-35be-41dc-9a38-abc160b429bd None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:45.655813 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:45.666958 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab1a3f-981c-49cf-acaa-14f78779769a None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:45.698260 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29106512-35be-41dc-9a38-abc160b429bd None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:45.699576 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:45.707578 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39da66-f7ea-4051-8940-df6a6ff098ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:45.708438 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39da66-f7ea-4051-8940-df6a6ff098ef None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:45.708718 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39da66-f7ea-4051-8940-df6a6ff098ef None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:45.708993 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39da66-f7ea-4051-8940-df6a6ff098ef None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:45.718523 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c39da66-f7ea-4051-8940-df6a6ff098ef None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:45.720009 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c39da66-f7ea-4051-8940-df6a6ff098ef None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:45.735321 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b225c0-6a83-4d6d-bafe-633fa9812b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:45.736275 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b225c0-6a83-4d6d-bafe-633fa9812b20 None admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:45.736838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b225c0-6a83-4d6d-bafe-633fa9812b20 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:45.737332 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b225c0-6a83-4d6d-bafe-633fa9812b20 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:45.737696 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-1c39da66-f7ea-4051-8940-df6a6ff098ef None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 01:44:45.737696 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:45.737696 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:45.737696 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:45.737696 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:45.737696 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:45.737696 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:45.737696 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:45.737696 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:45.737696 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:45.737696 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:45.737696 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:45.737696 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 01:44:45.737696 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 01:44:45.737696 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 01:44:45.737696 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 01:44:45.738623 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:45.738623 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:45.738623 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 01:44:45.738623 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:45.738623 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 01:44:45.738623 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:45.738623 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:45.738623 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:45.738623 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:45.738623 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:45.738623 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:45.738623 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:45.738623 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:45.738623 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:45.738623 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:45.738623 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:45.738623 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:45.738623 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:45.739594 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:45.739594 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:45.739594 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 01:44:45.739594 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 01:44:45.739594 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 01:44:45.739594 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 01:44:45.739594 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 01:44:45.739594 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 01:44:45.739594 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 01:44:45.739594 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:45.739594 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 467/937] 10.222.0.26 () {56 vars in 1117 bytes} [Thu Jun 23 01:44:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:45.741039 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b225c0-6a83-4d6d-bafe-633fa9812b20 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:45.742433 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b225c0-6a83-4d6d-bafe-633fa9812b20 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:45.750861 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39da66-f7ea-4051-8940-df6a6ff098ef None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:45.771643 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 471/938] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Jun 23 01:44:45 2022] POST /identity/v3/endpoints => generated 347 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:44:45.858283 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39da66-f7ea-4051-8940-df6a6ff098ef None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:45.901711 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036c09ed-1839-4b77-8599-f605784f8f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:45.902667 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c09ed-1839-4b77-8599-f605784f8f59 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:45.903127 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c09ed-1839-4b77-8599-f605784f8f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:45.903642 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036c09ed-1839-4b77-8599-f605784f8f59 None admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:45.905116 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036c09ed-1839-4b77-8599-f605784f8f59 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:45.905593 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036c09ed-1839-4b77-8599-f605784f8f59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:45.907876 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036c09ed-1839-4b77-8599-f605784f8f59 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:45.928225 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 468/939] 10.222.0.26 () {56 vars in 1149 bytes} [Thu Jun 23 01:44:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:45.934171 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b225c0-6a83-4d6d-bafe-633fa9812b20 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:46.039909 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b225c0-6a83-4d6d-bafe-633fa9812b20 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:46.057718 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:46.077742 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b3ad83-92dd-492c-bf99-611bd1cf844f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:46.078548 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b3ad83-92dd-492c-bf99-611bd1cf844f None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:46.078823 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b3ad83-92dd-492c-bf99-611bd1cf844f None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:46.079088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b3ad83-92dd-492c-bf99-611bd1cf844f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:46.091291 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b3ad83-92dd-492c-bf99-611bd1cf844f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:46.092763 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b3ad83-92dd-492c-bf99-611bd1cf844f None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:46.110568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-11b3ad83-92dd-492c-bf99-611bd1cf844f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:46.110568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:46.110568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:46.110568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:46.110568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:46.110568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:46.110568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:46.110568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:46.110568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:46.110568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:46.110568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:46.110568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:46.110568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:46.110568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:46.110568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:46.111670 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:46.111670 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:46.111670 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:46.111670 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:46.111670 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:46.111670 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:46.111670 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:46.111670 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:46.111670 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:46.111670 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:46.111670 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:46.111670 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:46.111670 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:46.111670 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:46.111670 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:46.111670 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:46.111670 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:46.113121 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:46.113121 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:46.113121 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:46.113121 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:46.113121 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 472/940] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:46.117826 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036c09ed-1839-4b77-8599-f605784f8f59 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:46.147636 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:46.222238 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036c09ed-1839-4b77-8599-f605784f8f59 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:46.256674 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdf3c2d-9ee1-4a31-9235-5201cae57261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:46.257665 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf3c2d-9ee1-4a31-9235-5201cae57261 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:46.258143 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf3c2d-9ee1-4a31-9235-5201cae57261 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:46.258688 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf3c2d-9ee1-4a31-9235-5201cae57261 None admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:46.259774 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdf3c2d-9ee1-4a31-9235-5201cae57261 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:46.260325 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdf3c2d-9ee1-4a31-9235-5201cae57261 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:46.261895 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdf3c2d-9ee1-4a31-9235-5201cae57261 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:46.274579 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 469/941] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:46.280428 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b3ad83-92dd-492c-bf99-611bd1cf844f None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:46.392216 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b3ad83-92dd-492c-bf99-611bd1cf844f None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:46.426716 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30197359-e154-4320-9ee9-be6635aad417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:46.427544 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30197359-e154-4320-9ee9-be6635aad417 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:46.427823 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30197359-e154-4320-9ee9-be6635aad417 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:46.428091 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30197359-e154-4320-9ee9-be6635aad417 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:46.435463 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30197359-e154-4320-9ee9-be6635aad417 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:46.436931 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30197359-e154-4320-9ee9-be6635aad417 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:46.451302 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-30197359-e154-4320-9ee9-be6635aad417 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:46.451302 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:46.451302 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:46.451302 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:46.451302 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:46.451302 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:46.451302 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:46.451302 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:46.451302 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:46.451302 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:46.451302 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:46.451302 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:46.451302 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 01:44:46.451302 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 01:44:46.451302 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 01:44:46.452222 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 01:44:46.452222 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:46.452222 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:46.452222 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:46.452222 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:46.452222 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:46.452222 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:46.452222 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:46.452222 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:46.452222 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:46.452222 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:46.452222 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:46.452222 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:46.452222 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:46.452222 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:46.452222 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:46.452222 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:46.453121 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 01:44:46.453121 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 01:44:46.453121 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 01:44:46.453121 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 01:44:46.453121 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 01:44:46.453121 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 01:44:46.453121 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:46.453121 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:46.453121 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 473/942] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:44:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:46.458592 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdf3c2d-9ee1-4a31-9235-5201cae57261 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:46.539355 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:46.561811 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdf3c2d-9ee1-4a31-9235-5201cae57261 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:46.596933 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a4097a-b4e5-4154-aa50-a37b825e1b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:46.597605 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a4097a-b4e5-4154-aa50-a37b825e1b44 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:46.597888 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a4097a-b4e5-4154-aa50-a37b825e1b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:46.598165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a4097a-b4e5-4154-aa50-a37b825e1b44 None admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:46.598965 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a4097a-b4e5-4154-aa50-a37b825e1b44 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:46.599302 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a4097a-b4e5-4154-aa50-a37b825e1b44 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:46.600755 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a4097a-b4e5-4154-aa50-a37b825e1b44 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:46.613837 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 470/943] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jun 23 01:44:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:46.619675 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30197359-e154-4320-9ee9-be6635aad417 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:46.651939 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:46.724476 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30197359-e154-4320-9ee9-be6635aad417 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:46.759354 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf6166c-d51e-4a09-af09-e8c448a6bfd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:46.760247 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6166c-d51e-4a09-af09-e8c448a6bfd0 None admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:46.760522 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6166c-d51e-4a09-af09-e8c448a6bfd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:46.760791 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6166c-d51e-4a09-af09-e8c448a6bfd0 None admin] PATH_INFO: `/v3/projects/a44a5e0e3e2d4b8e9b9cec7b5b5c725c/users/168e6181ad22460990f1f7361b483364/roles/f173bf8d88d34d82881f578a6c10a7ce` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:46.763022 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf6166c-d51e-4a09-af09-e8c448a6bfd0 None admin] RBAC: Authorizing `identity:create_grant(project_id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c, user_id=168e6181ad22460990f1f7361b483364, role_id=f173bf8d88d34d82881f578a6c10a7ce)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:46.764496 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf6166c-d51e-4a09-af09-e8c448a6bfd0 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:46.780832 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 474/944] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jun 23 01:44:46 2022] PUT /identity/v3/projects/a44a5e0e3e2d4b8e9b9cec7b5b5c725c/users/168e6181ad22460990f1f7361b483364/roles/f173bf8d88d34d82881f578a6c10a7ce => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:44:47.000247 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e35a53-54e1-470c-a487-09d071d4fa22 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:47.000806 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e35a53-54e1-470c-a487-09d071d4fa22 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:47.000806 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e35a53-54e1-470c-a487-09d071d4fa22 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:47.002884 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 471/945] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:47.009418 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9a527-7b0c-4ff0-ad13-d700d054585b None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:47.010114 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9a527-7b0c-4ff0-ad13-d700d054585b None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:47.010114 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9a527-7b0c-4ff0-ad13-d700d054585b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:47.027735 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-2dd9a527-7b0c-4ff0-ad13-d700d054585b None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:47.028936 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-2dd9a527-7b0c-4ff0-ad13-d700d054585b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:47.056811 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd6c0ee-508c-4756-a88e-ec47eab3f1bd None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:47.057442 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd6c0ee-508c-4756-a88e-ec47eab3f1bd None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:47.057908 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd6c0ee-508c-4756-a88e-ec47eab3f1bd None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:47.060679 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 472/946] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:47.064950 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc6624-53ee-45df-8292-2b6550a9fe03 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:47.065457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc6624-53ee-45df-8292-2b6550a9fe03 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:47.065938 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc6624-53ee-45df-8292-2b6550a9fe03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:47.082956 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-52bc6624-53ee-45df-8292-2b6550a9fe03 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:47.084502 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-52bc6624-53ee-45df-8292-2b6550a9fe03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:47.111248 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 475/947] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:44:47.130044 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd9a527-7b0c-4ff0-ad13-d700d054585b None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:47.158138 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:47.166269 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 473/948] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 01:44:47.189253 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bc6624-53ee-45df-8292-2b6550a9fe03 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:47.252159 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd9a527-7b0c-4ff0-ad13-d700d054585b None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:47.260751 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:47.298222 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95df1e2-f4bc-40d7-a002-8dc599dfeccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:47.299253 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95df1e2-f4bc-40d7-a002-8dc599dfeccc None admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:47.299704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95df1e2-f4bc-40d7-a002-8dc599dfeccc None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:47.300140 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95df1e2-f4bc-40d7-a002-8dc599dfeccc None admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:47.301015 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95df1e2-f4bc-40d7-a002-8dc599dfeccc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:47.301403 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bc6624-53ee-45df-8292-2b6550a9fe03 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:47.302805 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95df1e2-f4bc-40d7-a002-8dc599dfeccc None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:47.304956 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: WARNING keystone.server.flask.common [None req-e95df1e2-f4bc-40d7-a002-8dc599dfeccc None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 23 01:44:47.329199 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.sql.core [None req-e95df1e2-f4bc-40d7-a002-8dc599dfeccc None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 23 01:44:47.329199 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 23 01:44:47.329199 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [parameters: {'id': '9e7067da129443fb986073baf24957bb', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 23 01:44:47.329199 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80460) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 23 01:44:47.342973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252b22d-93b7-4e83-a0d2-6628f7541e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:47.343942 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252b22d-93b7-4e83-a0d2-6628f7541e8c None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:47.344362 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252b22d-93b7-4e83-a0d2-6628f7541e8c None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:47.344729 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252b22d-93b7-4e83-a0d2-6628f7541e8c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:47.345529 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8252b22d-93b7-4e83-a0d2-6628f7541e8c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:47.347044 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8252b22d-93b7-4e83-a0d2-6628f7541e8c None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:47.352084 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-e95df1e2-f4bc-40d7-a002-8dc599dfeccc None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9e7067da129443fb986073baf24957bb.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9e7067da129443fb986073baf24957bb. Jun 23 01:44:47.352084 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:47.352084 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 01:44:47.352084 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 01:44:47.352084 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 01:44:47.352084 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 01:44:47.352084 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 01:44:47.352084 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 01:44:47.352084 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 01:44:47.352084 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application conn.query(q) Jun 23 01:44:47.352084 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 01:44:47.352084 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 01:44:47.352084 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 01:44:47.352900 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application result.read() Jun 23 01:44:47.352900 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 01:44:47.352900 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 01:44:47.352900 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 01:44:47.352900 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 01:44:47.352900 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 01:44:47.352900 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 01:44:47.352900 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 01:44:47.352900 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 01:44:47.352900 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 23 01:44:47.352900 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:47.352900 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 23 01:44:47.352900 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:47.352900 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:47.352900 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 23 01:44:47.352900 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 23 01:44:47.352900 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 23 01:44:47.352900 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 23 01:44:47.353839 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 01:44:47.353839 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application next(self.gen) Jun 23 01:44:47.353839 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 23 01:44:47.353839 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application yield resource Jun 23 01:44:47.353839 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 01:44:47.353839 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application next(self.gen) Jun 23 01:44:47.353839 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 23 01:44:47.353839 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application self.session.rollback() Jun 23 01:44:47.353839 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 23 01:44:47.353839 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application self.force_reraise() Jun 23 01:44:47.353839 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 23 01:44:47.353839 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise self.value Jun 23 01:44:47.353839 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 23 01:44:47.353839 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 23 01:44:47.353839 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 23 01:44:47.353839 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application session.commit() Jun 23 01:44:47.353839 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 23 01:44:47.354734 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 23 01:44:47.354734 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 23 01:44:47.354734 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application self._prepare_impl() Jun 23 01:44:47.354734 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 23 01:44:47.354734 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application self.session.flush() Jun 23 01:44:47.354734 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 23 01:44:47.354734 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application self._flush(objects) Jun 23 01:44:47.354734 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 23 01:44:47.354734 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 23 01:44:47.354734 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 23 01:44:47.354734 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application compat.raise_( Jun 23 01:44:47.354734 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 01:44:47.354734 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception Jun 23 01:44:47.354734 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 23 01:44:47.354734 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application flush_context.execute() Jun 23 01:44:47.354734 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 23 01:44:47.354734 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rec.execute(self) Jun 23 01:44:47.355656 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 23 01:44:47.355656 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 23 01:44:47.355656 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 23 01:44:47.355656 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 23 01:44:47.355656 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 23 01:44:47.355656 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 23 01:44:47.355656 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 23 01:44:47.355656 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 23 01:44:47.355656 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 23 01:44:47.355656 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 23 01:44:47.355656 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 23 01:44:47.355656 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 23 01:44:47.355656 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 23 01:44:47.355656 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 23 01:44:47.355656 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 23 01:44:47.356451 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 23 01:44:47.356451 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 01:44:47.356451 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception Jun 23 01:44:47.356451 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 01:44:47.356451 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 01:44:47.356451 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 01:44:47.356451 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 01:44:47.356451 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 01:44:47.356451 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 01:44:47.356451 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 01:44:47.356451 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application conn.query(q) Jun 23 01:44:47.356451 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 01:44:47.356451 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 01:44:47.356451 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 01:44:47.356451 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application result.read() Jun 23 01:44:47.356451 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 01:44:47.356451 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 01:44:47.357290 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 01:44:47.357290 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 01:44:47.357290 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 01:44:47.357290 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 01:44:47.357290 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 01:44:47.357290 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 01:44:47.357290 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 23 01:44:47.357290 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 23 01:44:47.357290 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [parameters: {'id': '9e7067da129443fb986073baf24957bb', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 23 01:44:47.357290 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 23 01:44:47.357290 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:47.357290 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 01:44:47.357290 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:47.357290 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:47.357290 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 23 01:44:47.358069 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 23 01:44:47.358069 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 23 01:44:47.358069 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 23 01:44:47.358069 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 23 01:44:47.358069 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:47.358069 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 01:44:47.358069 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:47.358069 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:47.358069 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:47.358069 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:47.358069 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:47.358069 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:47.358069 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:47.358069 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:47.358069 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:47.358069 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:47.358069 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:47.358963 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:47.358963 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 23 01:44:47.358963 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 23 01:44:47.358963 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:47.358963 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:47.358963 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 23 01:44:47.358963 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 23 01:44:47.358963 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9e7067da129443fb986073baf24957bb. Jun 23 01:44:47.358963 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:47.361262 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-8252b22d-93b7-4e83-a0d2-6628f7541e8c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 01:44:47.361262 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:47.361262 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 01:44:47.361262 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 01:44:47.361262 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 01:44:47.361262 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 01:44:47.361262 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 01:44:47.361262 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:44:47.361262 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 01:44:47.361262 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:47.361262 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 01:44:47.361262 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:47.361262 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:47.361262 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:47.361262 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:47.361262 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:47.361262 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:47.362303 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:47.362303 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:47.362303 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:47.362303 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:47.362303 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:47.362303 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:47.362303 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 01:44:47.362303 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 01:44:47.362303 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 01:44:47.362303 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 01:44:47.362303 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:47.362303 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:47.362303 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:47.362303 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:47.362303 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:47.362303 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:47.363651 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:47.363651 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:47.363651 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:47.363651 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:47.363651 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:47.363651 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:47.363651 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:47.363651 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:47.363651 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:47.363651 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:47.363651 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 01:44:47.363651 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:44:47.363651 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 01:44:47.363651 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:47.365203 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 476/949] 10.222.0.26 () {60 vars in 1140 bytes} [Thu Jun 23 01:44:47 2022] POST /identity/v3/projects => generated 262 bytes in 235 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:44:47.369286 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 474/950] 10.222.0.26 () {56 vars in 1126 bytes} [Thu Jun 23 01:44:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:44:47.371281 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95df1e2-f4bc-40d7-a002-8dc599dfeccc None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:47.374806 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252b22d-93b7-4e83-a0d2-6628f7541e8c None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:47.466420 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:47.483100 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252b22d-93b7-4e83-a0d2-6628f7541e8c None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:47.495622 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95df1e2-f4bc-40d7-a002-8dc599dfeccc None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:47.513293 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:47.518896 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769951b5-2616-430b-adc1-2d5ae2fa2906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:47.520110 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769951b5-2616-430b-adc1-2d5ae2fa2906 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:47.521011 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769951b5-2616-430b-adc1-2d5ae2fa2906 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:47.521154 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769951b5-2616-430b-adc1-2d5ae2fa2906 None admin] PATH_INFO: `/v3/services` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:47.522181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769951b5-2616-430b-adc1-2d5ae2fa2906 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:47.522617 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769951b5-2616-430b-adc1-2d5ae2fa2906 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:47.524104 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769951b5-2616-430b-adc1-2d5ae2fa2906 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:47.534164 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 475/951] 10.222.0.26 () {56 vars in 1122 bytes} [Thu Jun 23 01:44:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:44:47.535148 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769951b5-2616-430b-adc1-2d5ae2fa2906 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:47.537679 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02024e7-e97c-4da2-835c-a0948d7339cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:47.538755 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02024e7-e97c-4da2-835c-a0948d7339cf None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:47.539325 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02024e7-e97c-4da2-835c-a0948d7339cf None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:47.539765 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02024e7-e97c-4da2-835c-a0948d7339cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:47.547727 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02024e7-e97c-4da2-835c-a0948d7339cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:47.549830 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02024e7-e97c-4da2-835c-a0948d7339cf None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:47.565971 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-c02024e7-e97c-4da2-835c-a0948d7339cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:47.565971 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:47.565971 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:47.565971 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:47.565971 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:47.565971 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:47.565971 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:47.565971 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:47.565971 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:47.565971 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:47.565971 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:47.565971 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:47.565971 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 01:44:47.565971 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 01:44:47.565971 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 01:44:47.566864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 01:44:47.566864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:47.566864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:47.566864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:47.566864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:47.566864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:47.566864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:47.566864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:47.566864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:47.566864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:47.566864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:47.566864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:47.566864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:47.566864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:47.566864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:47.566864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:47.566864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:47.567928 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 01:44:47.567928 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 01:44:47.567928 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 01:44:47.567928 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 01:44:47.567928 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 01:44:47.567928 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 01:44:47.567928 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:47.567928 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:47.570300 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 477/952] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:44:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:44:47.576079 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02024e7-e97c-4da2-835c-a0948d7339cf None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:47.662388 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769951b5-2616-430b-adc1-2d5ae2fa2906 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:47.701872 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f5b841-cc72-4ca6-a698-63912b198137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:47.702920 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f5b841-cc72-4ca6-a698-63912b198137 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:47.703441 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f5b841-cc72-4ca6-a698-63912b198137 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:47.703946 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f5b841-cc72-4ca6-a698-63912b198137 None admin] PATH_INFO: `/v3/services` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:47.704872 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f5b841-cc72-4ca6-a698-63912b198137 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:47.705429 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f5b841-cc72-4ca6-a698-63912b198137 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:47.705917 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02024e7-e97c-4da2-835c-a0948d7339cf None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:47.707246 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f5b841-cc72-4ca6-a698-63912b198137 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:47.717463 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 476/953] 10.222.0.26 () {56 vars in 1122 bytes} [Thu Jun 23 01:44:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:47.718596 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f5b841-cc72-4ca6-a698-63912b198137 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:47.741627 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde81524-93da-4492-885b-0228da4b9079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:47.742283 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde81524-93da-4492-885b-0228da4b9079 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:47.742552 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde81524-93da-4492-885b-0228da4b9079 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:47.742820 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde81524-93da-4492-885b-0228da4b9079 None admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:47.743710 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde81524-93da-4492-885b-0228da4b9079 None admin] RBAC: Adding query filter params (name=service) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:47.744021 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde81524-93da-4492-885b-0228da4b9079 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:47.745519 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde81524-93da-4492-885b-0228da4b9079 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:47.749176 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:47.775332 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 478/954] 10.222.0.26 () {56 vars in 1112 bytes} [Thu Jun 23 01:44:47 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:47.795636 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:47.841317 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f5b841-cc72-4ca6-a698-63912b198137 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:47.881015 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc3cdd-1767-4ee7-963d-8e09b71707e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:47.881699 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc3cdd-1767-4ee7-963d-8e09b71707e5 None admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:47.881977 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc3cdd-1767-4ee7-963d-8e09b71707e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:47.882250 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc3cdd-1767-4ee7-963d-8e09b71707e5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:47.883017 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fc3cdd-1767-4ee7-963d-8e09b71707e5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:47.884512 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fc3cdd-1767-4ee7-963d-8e09b71707e5 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:47.908695 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 477/955] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Jun 23 01:44:47 2022] POST /identity/v3/endpoints => generated 349 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:44:47.947075 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cc053-67ab-4ee7-af30-5bb7d1b835b1 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:47.947450 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cc053-67ab-4ee7-af30-5bb7d1b835b1 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:47.947691 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8cc053-67ab-4ee7-af30-5bb7d1b835b1 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:47.949469 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 479/956] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:47.955384 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f133322-3df6-4c8b-b5cb-37119ecb133a None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:47.955675 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f133322-3df6-4c8b-b5cb-37119ecb133a None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:47.955957 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f133322-3df6-4c8b-b5cb-37119ecb133a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:47.978384 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-7f133322-3df6-4c8b-b5cb-37119ecb133a None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:47.979654 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-7f133322-3df6-4c8b-b5cb-37119ecb133a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:48.066302 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 478/957] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:44:48.089982 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8cc053-67ab-4ee7-af30-5bb7d1b835b1 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:48.195202 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8cc053-67ab-4ee7-af30-5bb7d1b835b1 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:48.229805 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f4b315-2e0f-4c31-abed-bbd82bc12387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:48.230610 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4b315-2e0f-4c31-abed-bbd82bc12387 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:48.230892 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4b315-2e0f-4c31-abed-bbd82bc12387 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:48.231166 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4b315-2e0f-4c31-abed-bbd82bc12387 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:48.231859 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f4b315-2e0f-4c31-abed-bbd82bc12387 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:48.233249 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f4b315-2e0f-4c31-abed-bbd82bc12387 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:48.245709 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-43f4b315-2e0f-4c31-abed-bbd82bc12387 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 01:44:48.245709 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:48.245709 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 01:44:48.245709 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 01:44:48.245709 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 01:44:48.245709 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 01:44:48.245709 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 01:44:48.245709 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:44:48.245709 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 01:44:48.245709 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:48.245709 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 01:44:48.245709 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:48.245709 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:48.245709 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:48.245709 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:48.245709 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:48.245709 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:48.246825 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:48.246825 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:48.246825 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:48.246825 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:48.246825 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:48.246825 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:48.246825 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 01:44:48.246825 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 01:44:48.246825 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 01:44:48.246825 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 01:44:48.246825 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:48.246825 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:48.246825 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:48.246825 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:48.246825 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:48.246825 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:48.247935 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:48.247935 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:48.247935 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:48.247935 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:48.247935 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:48.247935 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:48.247935 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:48.247935 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:48.247935 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:48.247935 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:48.247935 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 01:44:48.247935 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:44:48.247935 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 01:44:48.247935 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:48.247935 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 480/958] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:44:48 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:48.257377 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f133322-3df6-4c8b-b5cb-37119ecb133a None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:48.301845 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:48.365109 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f133322-3df6-4c8b-b5cb-37119ecb133a None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:48.401887 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98993bc-2392-4f84-bf6d-43902c1efbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:48.402560 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98993bc-2392-4f84-bf6d-43902c1efbb8 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:48.402838 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98993bc-2392-4f84-bf6d-43902c1efbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:48.403110 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98993bc-2392-4f84-bf6d-43902c1efbb8 None admin] PATH_INFO: `/v3/services` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:48.403856 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98993bc-2392-4f84-bf6d-43902c1efbb8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:48.404158 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98993bc-2392-4f84-bf6d-43902c1efbb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:48.405522 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98993bc-2392-4f84-bf6d-43902c1efbb8 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:48.415322 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 479/959] 10.222.0.26 () {56 vars in 1112 bytes} [Thu Jun 23 01:44:48 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:48.416694 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f4b315-2e0f-4c31-abed-bbd82bc12387 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:48.479238 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:48.521268 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f4b315-2e0f-4c31-abed-bbd82bc12387 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:48.555520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c1cd35-df8d-4c27-bf59-0d625024d7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:48.556183 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c1cd35-df8d-4c27-bf59-0d625024d7b7 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:48.556461 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c1cd35-df8d-4c27-bf59-0d625024d7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:48.556735 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c1cd35-df8d-4c27-bf59-0d625024d7b7 None admin] PATH_INFO: `/v3/services` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:48.557448 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c1cd35-df8d-4c27-bf59-0d625024d7b7 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:48.557749 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c1cd35-df8d-4c27-bf59-0d625024d7b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:48.559127 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c1cd35-df8d-4c27-bf59-0d625024d7b7 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:48.571403 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 481/960] 10.222.0.26 () {56 vars in 1112 bytes} [Thu Jun 23 01:44:48 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:44:48.789827 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2191bf8e-9971-4d08-967c-5a0e07d0f861 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:48.790210 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2191bf8e-9971-4d08-967c-5a0e07d0f861 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:48.790407 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2191bf8e-9971-4d08-967c-5a0e07d0f861 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:48.792066 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 480/961] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:48.797481 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252eee97-9e85-4c6e-8576-0dd19cc10f95 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:48.797772 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252eee97-9e85-4c6e-8576-0dd19cc10f95 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:48.798051 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252eee97-9e85-4c6e-8576-0dd19cc10f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:48.815679 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-252eee97-9e85-4c6e-8576-0dd19cc10f95 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:48.817171 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-252eee97-9e85-4c6e-8576-0dd19cc10f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:48.891319 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 482/962] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:44:48.910027 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2191bf8e-9971-4d08-967c-5a0e07d0f861 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:49.025461 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2191bf8e-9971-4d08-967c-5a0e07d0f861 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:49.034943 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:49.065571 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e08c76-7d6b-459e-ad83-62c118690cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:49.066363 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e08c76-7d6b-459e-ad83-62c118690cde None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:49.066901 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e08c76-7d6b-459e-ad83-62c118690cde None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:49.066984 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e08c76-7d6b-459e-ad83-62c118690cde None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:49.077968 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e08c76-7d6b-459e-ad83-62c118690cde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:44:49.086413 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-d9e08c76-7d6b-459e-ad83-62c118690cde None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 01:44:49.086413 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:49.086413 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:49.086413 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:49.086413 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:49.086413 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:49.086413 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:49.086413 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:49.086413 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:49.086413 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:49.086413 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:49.086413 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:49.086413 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:44:49.086413 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:44:49.086413 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:44:49.086413 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:44:49.087383 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:44:49.087383 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:44:49.087383 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:44:49.087383 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:44:49.087383 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:49.087383 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:49.087383 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:49.087383 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:49.087383 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:49.087383 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:49.087383 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:49.087383 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:49.087383 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:49.087383 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:49.087383 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:49.087383 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:49.088222 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:49.088222 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:49.088222 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:49.088222 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:49.088222 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:44:49.088222 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:44:49.088222 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:44:49.088222 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:44:49.088222 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:44:49.088222 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:44:49.088222 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 01:44:49.088222 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:49.092590 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 481/963] 10.222.0.26 () {56 vars in 1120 bytes} [Thu Jun 23 01:44:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:49.093959 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252eee97-9e85-4c6e-8576-0dd19cc10f95 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:49.155615 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:49.196729 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252eee97-9e85-4c6e-8576-0dd19cc10f95 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:49.231560 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d60238-ef43-4548-b130-79120df3e54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:49.232263 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d60238-ef43-4548-b130-79120df3e54c None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:49.232543 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d60238-ef43-4548-b130-79120df3e54c None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:49.232815 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d60238-ef43-4548-b130-79120df3e54c None admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:49.233563 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d60238-ef43-4548-b130-79120df3e54c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:49.233865 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d60238-ef43-4548-b130-79120df3e54c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:49.235266 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d60238-ef43-4548-b130-79120df3e54c None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:49.247338 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 483/964] 10.222.0.26 () {56 vars in 1115 bytes} [Thu Jun 23 01:44:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:49.249275 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e08c76-7d6b-459e-ad83-62c118690cde None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:49.348906 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e08c76-7d6b-459e-ad83-62c118690cde None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:49.381831 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648ef497-37d1-4840-bc85-f1669f6876a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:49.382628 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ef497-37d1-4840-bc85-f1669f6876a5 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:49.382907 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ef497-37d1-4840-bc85-f1669f6876a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:49.383182 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ef497-37d1-4840-bc85-f1669f6876a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:49.396011 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648ef497-37d1-4840-bc85-f1669f6876a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:49.397460 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648ef497-37d1-4840-bc85-f1669f6876a5 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:49.414777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-648ef497-37d1-4840-bc85-f1669f6876a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:49.414777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:49.414777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:49.414777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:49.414777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:49.414777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:49.414777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:49.414777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:49.414777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:49.414777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:49.414777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:49.414777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:49.414777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:49.414777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:49.414777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:49.416787 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:49.416787 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:49.416787 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:49.416787 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:49.416787 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:49.416787 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:49.416787 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:49.416787 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:49.416787 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:49.416787 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:49.416787 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:49.416787 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:49.416787 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:49.416787 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:49.416787 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:49.416787 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:49.416787 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:49.418813 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:49.418813 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:49.418813 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:49.418813 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:49.423232 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 482/965] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:49.423411 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d60238-ef43-4548-b130-79120df3e54c None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:49.522486 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d60238-ef43-4548-b130-79120df3e54c None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:49.539164 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:49.557966 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a9813-5c11-4060-9637-98d6ae1baa3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:49.558673 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a9813-5c11-4060-9637-98d6ae1baa3d None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:49.558951 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a9813-5c11-4060-9637-98d6ae1baa3d None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:49.559238 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a9813-5c11-4060-9637-98d6ae1baa3d None admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:49.559979 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a9813-5c11-4060-9637-98d6ae1baa3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:49.560283 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a9813-5c11-4060-9637-98d6ae1baa3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:49.561668 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a9813-5c11-4060-9637-98d6ae1baa3d None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:49.578557 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 484/966] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:49.580125 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648ef497-37d1-4840-bc85-f1669f6876a5 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:49.603654 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:49.684023 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648ef497-37d1-4840-bc85-f1669f6876a5 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:49.719127 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b415521-f804-4085-8fea-d6c51ca8d1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:49.719980 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b415521-f804-4085-8fea-d6c51ca8d1c4 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:49.720258 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b415521-f804-4085-8fea-d6c51ca8d1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:49.720529 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b415521-f804-4085-8fea-d6c51ca8d1c4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:49.728326 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b415521-f804-4085-8fea-d6c51ca8d1c4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:49.729795 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b415521-f804-4085-8fea-d6c51ca8d1c4 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:49.744546 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-2b415521-f804-4085-8fea-d6c51ca8d1c4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 01:44:49.744546 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:49.744546 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:49.744546 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:49.744546 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:49.744546 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:49.744546 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:49.744546 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:49.744546 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:49.744546 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:49.744546 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:49.744546 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:49.744546 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 01:44:49.744546 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 01:44:49.744546 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 01:44:49.744546 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 01:44:49.746648 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:49.746648 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:49.746648 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 01:44:49.746648 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:49.746648 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 01:44:49.746648 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:49.746648 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:49.746648 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:49.746648 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:49.746648 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:49.746648 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:49.746648 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:49.746648 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:49.746648 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:49.746648 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:49.746648 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:49.746648 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:49.746648 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:49.748839 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:49.748839 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:49.748839 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 01:44:49.748839 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 01:44:49.748839 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 01:44:49.748839 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 01:44:49.748839 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 01:44:49.748839 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 01:44:49.748839 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 01:44:49.748839 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:49.751256 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 483/967] 10.222.0.26 () {56 vars in 1117 bytes} [Thu Jun 23 01:44:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:49.752717 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a9813-5c11-4060-9637-98d6ae1baa3d None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:49.854156 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a9813-5c11-4060-9637-98d6ae1baa3d None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:49.888524 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864849c-bd73-46d4-a874-058c1d6c46e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:49.889411 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864849c-bd73-46d4-a874-058c1d6c46e1 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:49.889873 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864849c-bd73-46d4-a874-058c1d6c46e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:49.890302 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864849c-bd73-46d4-a874-058c1d6c46e1 None admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:49.891438 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2864849c-bd73-46d4-a874-058c1d6c46e1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:49.891883 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2864849c-bd73-46d4-a874-058c1d6c46e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:49.893480 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2864849c-bd73-46d4-a874-058c1d6c46e1 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:49.915310 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 485/968] 10.222.0.26 () {56 vars in 1149 bytes} [Thu Jun 23 01:44:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:49.917817 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b415521-f804-4085-8fea-d6c51ca8d1c4 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:49.998151 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:50.000245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b065495-1795-4a5c-8140-770818204c26 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:50.000535 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b065495-1795-4a5c-8140-770818204c26 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:50.000814 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b065495-1795-4a5c-8140-770818204c26 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:50.003261 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 486/969] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:50.008596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6dd29-b14d-4eff-82c2-a1b7062b505c None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:50.008883 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6dd29-b14d-4eff-82c2-a1b7062b505c None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:50.009162 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6dd29-b14d-4eff-82c2-a1b7062b505c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:50.025193 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b415521-f804-4085-8fea-d6c51ca8d1c4 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:50.027441 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-1ba6dd29-b14d-4eff-82c2-a1b7062b505c None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:50.028832 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-1ba6dd29-b14d-4eff-82c2-a1b7062b505c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:50.057705 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:50.066285 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a788ffef-4779-4f49-81aa-7e73dbf5c547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:50.067136 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a788ffef-4779-4f49-81aa-7e73dbf5c547 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:50.067725 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a788ffef-4779-4f49-81aa-7e73dbf5c547 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:50.067725 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a788ffef-4779-4f49-81aa-7e73dbf5c547 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:50.101796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 487/970] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 01:44:50.103704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3365a6-e0bf-4755-80df-998248fb5e39 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:50.103993 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3365a6-e0bf-4755-80df-998248fb5e39 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:50.104320 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3365a6-e0bf-4755-80df-998248fb5e39 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:50.105687 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 488/971] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:50.111167 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7365c00-4069-424e-a44e-87b006b2dcfe None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:50.111505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7365c00-4069-424e-a44e-87b006b2dcfe None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:50.111778 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7365c00-4069-424e-a44e-87b006b2dcfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:50.129121 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-b7365c00-4069-424e-a44e-87b006b2dcfe None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:50.130448 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-b7365c00-4069-424e-a44e-87b006b2dcfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:50.147020 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a788ffef-4779-4f49-81aa-7e73dbf5c547 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:50.148539 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a788ffef-4779-4f49-81aa-7e73dbf5c547 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:50.203394 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 489/972] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 01:44:50.203394 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7365c00-4069-424e-a44e-87b006b2dcfe None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:50.235122 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:50.264297 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 484/973] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:44:49 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 01:44:50.265233 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a788ffef-4779-4f49-81aa-7e73dbf5c547 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:50.313669 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:50.321724 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7365c00-4069-424e-a44e-87b006b2dcfe None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:50.357742 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41721c2d-35ca-49e9-a198-0fd0dd496e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:50.358342 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41721c2d-35ca-49e9-a198-0fd0dd496e02 None admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:50.358622 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41721c2d-35ca-49e9-a198-0fd0dd496e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:50.358890 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41721c2d-35ca-49e9-a198-0fd0dd496e02 None admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:50.360053 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41721c2d-35ca-49e9-a198-0fd0dd496e02 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:50.361610 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41721c2d-35ca-49e9-a198-0fd0dd496e02 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:50.371688 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a788ffef-4779-4f49-81aa-7e73dbf5c547 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:50.376057 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.sql.core [None req-41721c2d-35ca-49e9-a198-0fd0dd496e02 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 23 01:44:50.376057 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 23 01:44:50.376057 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [parameters: {'user_id': '908cdb58adb146fe9f4b4da110dc3e14', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 23 01:44:50.376057 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80460) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 23 01:44:50.383851 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-41721c2d-35ca-49e9-a198-0fd0dd496e02 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 23 01:44:50.383851 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:50.383851 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 01:44:50.383851 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 01:44:50.383851 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 01:44:50.383851 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 01:44:50.383851 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 01:44:50.383851 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 01:44:50.383851 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 01:44:50.383851 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application conn.query(q) Jun 23 01:44:50.383851 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 01:44:50.383851 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 01:44:50.383851 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 01:44:50.383851 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application result.read() Jun 23 01:44:50.383851 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 01:44:50.384732 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 01:44:50.384732 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 01:44:50.384732 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 01:44:50.384732 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 01:44:50.384732 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 01:44:50.384732 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 01:44:50.384732 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 01:44:50.384732 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 23 01:44:50.384732 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:50.384732 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 23 01:44:50.384732 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:50.384732 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:50.384732 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 23 01:44:50.384732 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 23 01:44:50.384732 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 23 01:44:50.384732 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 23 01:44:50.384732 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 01:44:50.384732 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application next(self.gen) Jun 23 01:44:50.385668 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 23 01:44:50.385668 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application yield resource Jun 23 01:44:50.385668 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 01:44:50.385668 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application next(self.gen) Jun 23 01:44:50.385668 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 23 01:44:50.385668 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application self.session.rollback() Jun 23 01:44:50.385668 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 23 01:44:50.385668 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application self.force_reraise() Jun 23 01:44:50.385668 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 23 01:44:50.385668 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise self.value Jun 23 01:44:50.385668 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 23 01:44:50.385668 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 23 01:44:50.385668 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 23 01:44:50.385668 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application session.commit() Jun 23 01:44:50.385668 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 23 01:44:50.385668 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 23 01:44:50.385668 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 23 01:44:50.387435 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application self._prepare_impl() Jun 23 01:44:50.387435 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 23 01:44:50.387435 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application self.session.flush() Jun 23 01:44:50.387435 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 23 01:44:50.387435 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application self._flush(objects) Jun 23 01:44:50.387435 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 23 01:44:50.387435 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 23 01:44:50.387435 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 23 01:44:50.387435 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application compat.raise_( Jun 23 01:44:50.387435 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 01:44:50.387435 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception Jun 23 01:44:50.387435 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 23 01:44:50.387435 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application flush_context.execute() Jun 23 01:44:50.387435 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 23 01:44:50.387435 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rec.execute(self) Jun 23 01:44:50.387435 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 23 01:44:50.387435 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 23 01:44:50.388379 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 23 01:44:50.388379 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 23 01:44:50.388379 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 23 01:44:50.388379 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 23 01:44:50.388379 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 23 01:44:50.388379 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 23 01:44:50.388379 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 23 01:44:50.388379 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 23 01:44:50.388379 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 23 01:44:50.388379 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 23 01:44:50.388379 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 23 01:44:50.388379 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 23 01:44:50.388379 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 23 01:44:50.388379 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 23 01:44:50.388379 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 01:44:50.388379 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception Jun 23 01:44:50.389224 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 01:44:50.389224 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 01:44:50.389224 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 01:44:50.389224 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 01:44:50.389224 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 01:44:50.389224 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 01:44:50.389224 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 01:44:50.389224 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application conn.query(q) Jun 23 01:44:50.389224 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 01:44:50.389224 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 01:44:50.389224 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 01:44:50.389224 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application result.read() Jun 23 01:44:50.389224 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 01:44:50.389224 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 01:44:50.389224 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 01:44:50.389224 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 01:44:50.391275 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 01:44:50.391275 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 01:44:50.391275 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 01:44:50.391275 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 01:44:50.391275 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 23 01:44:50.391275 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 23 01:44:50.391275 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [parameters: {'user_id': '908cdb58adb146fe9f4b4da110dc3e14', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 23 01:44:50.391275 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 23 01:44:50.391275 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:50.391275 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 01:44:50.391275 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:50.391275 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:50.391275 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:50.391275 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:50.391275 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:50.393394 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:50.393394 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:50.393394 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:50.393394 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:50.393394 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:50.393394 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:50.393394 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:50.393394 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 23 01:44:50.393394 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 23 01:44:50.393394 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:50.393394 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:50.393394 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 01:44:50.393394 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:50.393394 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 01:44:50.393394 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:50.393394 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 23 01:44:50.393394 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 23 01:44:50.395530 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 23 01:44:50.395530 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 23 01:44:50.395530 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 23 01:44:50.395530 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 23 01:44:50.395530 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 23 01:44:50.395530 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:50.395530 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 490/974] 10.222.0.26 () {60 vars in 1131 bytes} [Thu Jun 23 01:44:50 2022] POST /identity/v3/users => generated 160 bytes in 186 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:50.395530 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41721c2d-35ca-49e9-a198-0fd0dd496e02 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:50.418663 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dc6f8b-21c7-4e7c-8363-023d7e5965d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:50.419503 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dc6f8b-21c7-4e7c-8363-023d7e5965d8 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:50.419782 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dc6f8b-21c7-4e7c-8363-023d7e5965d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:50.420049 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dc6f8b-21c7-4e7c-8363-023d7e5965d8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:50.420713 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dc6f8b-21c7-4e7c-8363-023d7e5965d8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:50.422084 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dc6f8b-21c7-4e7c-8363-023d7e5965d8 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:50.435845 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-f8dc6f8b-21c7-4e7c-8363-023d7e5965d8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 01:44:50.435845 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:50.435845 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 01:44:50.435845 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 01:44:50.435845 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 01:44:50.435845 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 01:44:50.435845 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 01:44:50.435845 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:44:50.435845 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 01:44:50.435845 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:50.435845 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 01:44:50.435845 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:50.435845 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:50.435845 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:50.435845 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:50.435845 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:50.435845 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:50.436883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:50.436883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:50.436883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:50.436883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:50.436883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:50.436883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:50.436883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 01:44:50.436883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 01:44:50.436883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 01:44:50.436883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 01:44:50.436883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:50.436883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:50.436883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:50.436883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:50.436883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:50.436883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:50.437707 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:50.437707 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:50.437707 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:50.437707 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:50.437707 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:50.437707 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:50.437707 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:50.437707 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:50.437707 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:50.437707 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:50.437707 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 01:44:50.437707 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:44:50.437707 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 01:44:50.437707 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:50.439042 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 485/975] 10.222.0.26 () {56 vars in 1126 bytes} [Thu Jun 23 01:44:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:50.439119 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dc6f8b-21c7-4e7c-8363-023d7e5965d8 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:50.491443 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:50.504731 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41721c2d-35ca-49e9-a198-0fd0dd496e02 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:50.542837 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b099852-a262-4d77-a961-26385094adb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:50.543768 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b099852-a262-4d77-a961-26385094adb4 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:50.544047 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b099852-a262-4d77-a961-26385094adb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:50.544317 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b099852-a262-4d77-a961-26385094adb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:50.552450 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dc6f8b-21c7-4e7c-8363-023d7e5965d8 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:50.556611 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b099852-a262-4d77-a961-26385094adb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:50.558273 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b099852-a262-4d77-a961-26385094adb4 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:50.576960 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-3b099852-a262-4d77-a961-26385094adb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:50.576960 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:50.576960 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:50.576960 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:50.576960 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:50.576960 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:50.576960 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:50.576960 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:50.576960 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:50.576960 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:50.576960 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:50.576960 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:50.576960 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:44:50.576960 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:44:50.576960 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:44:50.577872 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:44:50.577872 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:50.577872 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:50.577872 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:50.577872 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:50.577872 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:50.577872 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:50.577872 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:50.577872 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:50.577872 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:50.577872 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:50.577872 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:50.577872 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:50.577872 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:50.577872 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:50.577872 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:50.577872 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:50.578758 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:44:50.578758 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:44:50.578758 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:44:50.578758 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:50.578758 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 491/976] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:50.579046 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b099852-a262-4d77-a961-26385094adb4 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:50.592014 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:50.592975 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dda962a-56c1-46f1-aea7-ec049f51dafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:50.593794 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dda962a-56c1-46f1-aea7-ec049f51dafc None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:50.594181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dda962a-56c1-46f1-aea7-ec049f51dafc None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:50.594340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dda962a-56c1-46f1-aea7-ec049f51dafc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:50.602925 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dda962a-56c1-46f1-aea7-ec049f51dafc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:50.604617 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dda962a-56c1-46f1-aea7-ec049f51dafc None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:50.618961 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-4dda962a-56c1-46f1-aea7-ec049f51dafc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 01:44:50.618961 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:50.618961 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:50.618961 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:50.618961 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:50.618961 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:50.618961 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:50.618961 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:50.618961 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:50.618961 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:50.618961 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:50.618961 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:50.618961 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 01:44:50.618961 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 01:44:50.618961 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 01:44:50.618961 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 01:44:50.620463 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:50.620463 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:50.620463 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 01:44:50.620463 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:50.620463 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 01:44:50.620463 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:44:50.620463 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:50.620463 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:50.620463 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:50.620463 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:50.620463 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:50.620463 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:50.620463 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:50.620463 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:50.620463 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:50.620463 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:50.620463 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:50.620463 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:50.623065 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:50.623065 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:50.623065 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 01:44:50.623065 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 01:44:50.623065 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 01:44:50.623065 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 01:44:50.623065 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 01:44:50.623065 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 01:44:50.623065 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 01:44:50.623065 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:50.623065 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 486/977] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:44:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:50.623065 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dda962a-56c1-46f1-aea7-ec049f51dafc None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:50.687717 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b099852-a262-4d77-a961-26385094adb4 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:50.719769 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dda962a-56c1-46f1-aea7-ec049f51dafc None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:50.720461 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2585047-86d0-4d7f-98dc-6ef81df73ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:50.721170 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2585047-86d0-4d7f-98dc-6ef81df73ac1 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:50.721375 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:50.721476 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2585047-86d0-4d7f-98dc-6ef81df73ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:50.721946 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2585047-86d0-4d7f-98dc-6ef81df73ac1 None admin] PATH_INFO: `/v3/services` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:50.722529 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2585047-86d0-4d7f-98dc-6ef81df73ac1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:50.722885 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2585047-86d0-4d7f-98dc-6ef81df73ac1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:50.724369 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2585047-86d0-4d7f-98dc-6ef81df73ac1 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:50.732188 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 492/978] 10.222.0.26 () {56 vars in 1122 bytes} [Thu Jun 23 01:44:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:50.733538 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dbd265-3969-4962-9914-207faed4113b None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:50.733861 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dbd265-3969-4962-9914-207faed4113b None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:50.734142 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dbd265-3969-4962-9914-207faed4113b None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:50.735531 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 493/979] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:50.736137 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dbd265-3969-4962-9914-207faed4113b None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:50.761857 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a565ce-ead2-4b09-b0ca-1b0e0021de20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:50.762513 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a565ce-ead2-4b09-b0ca-1b0e0021de20 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:50.762790 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a565ce-ead2-4b09-b0ca-1b0e0021de20 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:50.763059 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a565ce-ead2-4b09-b0ca-1b0e0021de20 None admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:50.763816 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a565ce-ead2-4b09-b0ca-1b0e0021de20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:50.764126 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a565ce-ead2-4b09-b0ca-1b0e0021de20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:50.765500 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a565ce-ead2-4b09-b0ca-1b0e0021de20 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:50.777334 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 487/980] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:44:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:50.778120 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a565ce-ead2-4b09-b0ca-1b0e0021de20 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:50.813800 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:50.844525 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dbd265-3969-4962-9914-207faed4113b None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:50.879428 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f10fa3-d6ce-4e0e-a778-2877c128a38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:50.880039 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f10fa3-d6ce-4e0e-a778-2877c128a38b None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:50.880354 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f10fa3-d6ce-4e0e-a778-2877c128a38b None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:50.880657 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f10fa3-d6ce-4e0e-a778-2877c128a38b None admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:50.881228 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a565ce-ead2-4b09-b0ca-1b0e0021de20 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:50.881490 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f10fa3-d6ce-4e0e-a778-2877c128a38b None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:50.881810 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f10fa3-d6ce-4e0e-a778-2877c128a38b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:50.883267 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f10fa3-d6ce-4e0e-a778-2877c128a38b None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:50.914568 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f38a3fe-adf5-44e2-bca2-46a3db878f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:50.915247 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38a3fe-adf5-44e2-bca2-46a3db878f1d None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:50.915529 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38a3fe-adf5-44e2-bca2-46a3db878f1d None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:50.915801 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38a3fe-adf5-44e2-bca2-46a3db878f1d None admin] PATH_INFO: `/v3/services` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:50.916511 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f38a3fe-adf5-44e2-bca2-46a3db878f1d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:50.916816 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f38a3fe-adf5-44e2-bca2-46a3db878f1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:50.918335 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 494/981] 10.222.0.26 () {56 vars in 1101 bytes} [Thu Jun 23 01:44:50 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:50.918449 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f38a3fe-adf5-44e2-bca2-46a3db878f1d None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:50.919695 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac0903-2369-40e2-9660-77e235f2a12e None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:50.919982 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac0903-2369-40e2-9660-77e235f2a12e None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:50.920301 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac0903-2369-40e2-9660-77e235f2a12e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:50.923585 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 488/982] 10.222.0.26 () {56 vars in 1122 bytes} [Thu Jun 23 01:44:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:50.924715 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f38a3fe-adf5-44e2-bca2-46a3db878f1d None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:50.953138 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-1dac0903-2369-40e2-9660-77e235f2a12e None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:50.954749 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-1dac0903-2369-40e2-9660-77e235f2a12e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:50.960342 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:51.031950 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 495/983] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:44:51.040364 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac0903-2369-40e2-9660-77e235f2a12e None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:51.042243 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f38a3fe-adf5-44e2-bca2-46a3db878f1d None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:51.081678 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e0a7c-708a-4e26-911b-4f4e9fd444d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:51.082741 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e0a7c-708a-4e26-911b-4f4e9fd444d9 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:51.083018 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e0a7c-708a-4e26-911b-4f4e9fd444d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:51.083324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e0a7c-708a-4e26-911b-4f4e9fd444d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:51.093482 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696e0a7c-708a-4e26-911b-4f4e9fd444d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:51.095677 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696e0a7c-708a-4e26-911b-4f4e9fd444d9 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:51.110749 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-696e0a7c-708a-4e26-911b-4f4e9fd444d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:51.110749 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:51.110749 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:51.110749 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:51.110749 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:51.110749 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:51.110749 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:51.110749 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:51.110749 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:51.110749 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:51.110749 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:51.110749 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:51.110749 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 01:44:51.110749 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 01:44:51.110749 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 01:44:51.111977 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 01:44:51.111977 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:51.111977 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:51.111977 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:51.111977 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:51.111977 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:51.111977 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:51.111977 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:51.111977 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:51.111977 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:51.111977 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:51.111977 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:51.111977 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:51.111977 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:51.111977 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:51.111977 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:51.111977 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:51.112883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 01:44:51.112883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 01:44:51.112883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 01:44:51.112883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 01:44:51.112883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 01:44:51.112883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 01:44:51.112883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:44:51.112883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:51.112883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 489/984] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:44:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:51.112883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e0a7c-708a-4e26-911b-4f4e9fd444d9 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:51.116704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:51.169722 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac0903-2369-40e2-9660-77e235f2a12e None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:51.202999 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57544dc0-36fa-4480-a1de-d81685bf5ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:51.203678 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57544dc0-36fa-4480-a1de-d81685bf5ada None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:51.203954 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57544dc0-36fa-4480-a1de-d81685bf5ada None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:51.204221 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57544dc0-36fa-4480-a1de-d81685bf5ada None admin] PATH_INFO: `/v3/endpoints` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:51.205076 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57544dc0-36fa-4480-a1de-d81685bf5ada None admin] RBAC: Adding query filter params (interface=admin, service_id=52022d5659834a0f8e19e5fded581053, region_id=RegionOne) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:51.205385 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57544dc0-36fa-4480-a1de-d81685bf5ada None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:51.206763 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57544dc0-36fa-4480-a1de-d81685bf5ada None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:51.211454 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:51.211711 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 496/985] 10.222.0.26 () {56 vars in 1249 bytes} [Thu Jun 23 01:44:51 2022] GET /identity/v3/endpoints?service_id=52022d5659834a0f8e19e5fded581053&interface=admin®ion_id=RegionOne => generated 193 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:51.212463 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57544dc0-36fa-4480-a1de-d81685bf5ada None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:51.226736 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e0a7c-708a-4e26-911b-4f4e9fd444d9 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:51.259663 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc237a8-4462-44d4-8687-424db476c6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:51.260318 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc237a8-4462-44d4-8687-424db476c6dc None admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:51.260596 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc237a8-4462-44d4-8687-424db476c6dc None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:51.260870 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc237a8-4462-44d4-8687-424db476c6dc None admin] PATH_INFO: `/v3/services` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:51.261587 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc237a8-4462-44d4-8687-424db476c6dc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:51.263020 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc237a8-4462-44d4-8687-424db476c6dc None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:51.276843 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 490/986] 10.222.0.26 () {60 vars in 1141 bytes} [Thu Jun 23 01:44:51 2022] POST /identity/v3/services => generated 242 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:44:51.277635 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc237a8-4462-44d4-8687-424db476c6dc None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:51.315643 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57544dc0-36fa-4480-a1de-d81685bf5ada None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:51.343170 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:51.362747 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388bbbbb-4391-4c83-8374-5902c8252ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:51.363450 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388bbbbb-4391-4c83-8374-5902c8252ff9 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:51.363734 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388bbbbb-4391-4c83-8374-5902c8252ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:51.364010 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388bbbbb-4391-4c83-8374-5902c8252ff9 None admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:51.364829 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388bbbbb-4391-4c83-8374-5902c8252ff9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:51.365144 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388bbbbb-4391-4c83-8374-5902c8252ff9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:51.366615 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388bbbbb-4391-4c83-8374-5902c8252ff9 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:51.380089 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 497/987] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jun 23 01:44:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:51.386287 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388bbbbb-4391-4c83-8374-5902c8252ff9 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:51.406382 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc237a8-4462-44d4-8687-424db476c6dc None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:51.445133 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0bea1e-0072-487b-852a-004899cfc97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:51.445808 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0bea1e-0072-487b-852a-004899cfc97f None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:51.446089 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0bea1e-0072-487b-852a-004899cfc97f None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:51.446364 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0bea1e-0072-487b-852a-004899cfc97f None admin] PATH_INFO: `/v3/services` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:51.447058 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0bea1e-0072-487b-852a-004899cfc97f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:51.448469 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0bea1e-0072-487b-852a-004899cfc97f None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:51.455833 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 491/988] 10.222.0.26 () {56 vars in 1087 bytes} [Thu Jun 23 01:44:51 2022] GET /identity/v3/services => generated 1727 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:44:51.507058 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388bbbbb-4391-4c83-8374-5902c8252ff9 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:51.545612 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a47cfe-0bc3-4523-9f65-fe92b736cc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:51.546281 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a47cfe-0bc3-4523-9f65-fe92b736cc04 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:51.546562 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a47cfe-0bc3-4523-9f65-fe92b736cc04 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:51.546838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a47cfe-0bc3-4523-9f65-fe92b736cc04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:51.547728 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a47cfe-0bc3-4523-9f65-fe92b736cc04 None admin] RBAC: Adding query filter params (role.id=f173bf8d88d34d82881f578a6c10a7ce, scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c, user.id=168e6181ad22460990f1f7361b483364) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:51.548042 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a47cfe-0bc3-4523-9f65-fe92b736cc04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:51.549592 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a47cfe-0bc3-4523-9f65-fe92b736cc04 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:51.556993 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 498/989] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:44:51 2022] GET /identity/v3/role_assignments?user.id=168e6181ad22460990f1f7361b483364&scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c&role.id=f173bf8d88d34d82881f578a6c10a7ce => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:53.331822 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500531e7-9447-4adc-b5ba-67cc0c57e28d None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:53.332346 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500531e7-9447-4adc-b5ba-67cc0c57e28d None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:53.332346 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500531e7-9447-4adc-b5ba-67cc0c57e28d None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:53.333889 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 492/990] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:53.339157 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066269c7-c37b-43ad-bbca-80127cd392f5 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:53.339599 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066269c7-c37b-43ad-bbca-80127cd392f5 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:53.339767 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066269c7-c37b-43ad-bbca-80127cd392f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:53.357622 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-066269c7-c37b-43ad-bbca-80127cd392f5 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:53.358815 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-066269c7-c37b-43ad-bbca-80127cd392f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:53.401388 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:53.430939 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 499/991] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:53 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:44:53.453919 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500531e7-9447-4adc-b5ba-67cc0c57e28d None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:53.467444 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:53.474425 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294b7de-a9b0-4646-8f2a-60fa39baec50 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:53.474786 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294b7de-a9b0-4646-8f2a-60fa39baec50 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:53.475070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294b7de-a9b0-4646-8f2a-60fa39baec50 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:53.476761 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 500/992] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 01:44:53.481626 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b15005-acb9-465a-94c1-ef4297a538b0 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:53.481920 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b15005-acb9-465a-94c1-ef4297a538b0 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:53.482254 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b15005-acb9-465a-94c1-ef4297a538b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:53.499237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-59b15005-acb9-465a-94c1-ef4297a538b0 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:53.500429 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-59b15005-acb9-465a-94c1-ef4297a538b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:53.564981 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500531e7-9447-4adc-b5ba-67cc0c57e28d None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:53.571377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 501/993] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:53 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 01:44:53.595004 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223f4c0-5566-4ca2-823e-b5093043e311 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:53.595327 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223f4c0-5566-4ca2-823e-b5093043e311 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:53.595655 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223f4c0-5566-4ca2-823e-b5093043e311 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:53.597015 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 502/994] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 01:44:53.597697 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223f4c0-5566-4ca2-823e-b5093043e311 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:53.601945 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b0387-d770-4e4d-8339-69752cdf6d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:53.602662 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b0387-d770-4e4d-8339-69752cdf6d40 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:53.602942 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b0387-d770-4e4d-8339-69752cdf6d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:53.603233 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b0387-d770-4e4d-8339-69752cdf6d40 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:53.603898 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366b0387-d770-4e4d-8339-69752cdf6d40 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:53.605275 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366b0387-d770-4e4d-8339-69752cdf6d40 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:53.617144 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-366b0387-d770-4e4d-8339-69752cdf6d40 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 01:44:53.617144 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:53.617144 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 01:44:53.617144 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 01:44:53.617144 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 01:44:53.617144 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 01:44:53.617144 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 01:44:53.617144 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:44:53.617144 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 01:44:53.617144 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:53.617144 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 01:44:53.617144 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:53.617144 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:53.617144 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:53.617144 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:53.617144 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:53.617144 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:53.618113 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:53.618113 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:53.618113 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:53.618113 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:53.618113 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:53.618113 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:53.618113 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 01:44:53.618113 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 01:44:53.618113 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 01:44:53.618113 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 01:44:53.618113 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:53.618113 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:53.618113 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:53.618113 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:53.618113 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:53.618113 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:53.618944 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:53.618944 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:53.618944 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:53.618944 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:53.618944 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:53.618944 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:53.618944 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:53.618944 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:53.618944 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:53.618944 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:53.618944 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 01:44:53.618944 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:44:53.618944 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 01:44:53.618944 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:53.618944 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 493/995] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:44:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:53.619781 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f1cda-c598-4139-a360-6505ffe6dfb5 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:53.619781 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f1cda-c598-4139-a360-6505ffe6dfb5 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:53.620238 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f1cda-c598-4139-a360-6505ffe6dfb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:53.636214 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-a75f1cda-c598-4139-a360-6505ffe6dfb5 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:53.637404 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-a75f1cda-c598-4139-a360-6505ffe6dfb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:53.704554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223f4c0-5566-4ca2-823e-b5093043e311 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:53.706138 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 494/996] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:53 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:44:53.707117 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f1cda-c598-4139-a360-6505ffe6dfb5 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:53.708013 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:53.710457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:53.742710 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a24da6-6224-4397-8f2e-6f2980001421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:53.743545 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a24da6-6224-4397-8f2e-6f2980001421 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:53.743828 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a24da6-6224-4397-8f2e-6f2980001421 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:53.744103 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a24da6-6224-4397-8f2e-6f2980001421 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:53.744763 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a24da6-6224-4397-8f2e-6f2980001421 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:53.746150 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a24da6-6224-4397-8f2e-6f2980001421 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:53.758231 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-c6a24da6-6224-4397-8f2e-6f2980001421 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 01:44:53.758231 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:53.758231 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 01:44:53.758231 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 01:44:53.758231 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 01:44:53.758231 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 01:44:53.758231 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 01:44:53.758231 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:44:53.758231 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 01:44:53.758231 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:53.758231 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 01:44:53.758231 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:53.758231 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:53.758231 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:53.758231 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:53.758231 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:53.758231 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:53.759414 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:53.759414 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:53.759414 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:53.759414 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:53.759414 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:53.759414 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:53.759414 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 01:44:53.759414 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 01:44:53.759414 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 01:44:53.759414 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 01:44:53.759414 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:53.759414 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:53.759414 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:53.759414 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:53.759414 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:53.759414 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:53.760441 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:53.760441 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:53.760441 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:53.760441 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:53.760441 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:53.760441 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:53.760441 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:53.760441 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:53.760441 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:53.760441 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:53.760441 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 01:44:53.760441 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:44:53.760441 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 01:44:53.760441 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:53.760441 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 503/997] 10.222.0.26 () {56 vars in 1126 bytes} [Thu Jun 23 01:44:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:53.761235 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a24da6-6224-4397-8f2e-6f2980001421 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:53.812729 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f1cda-c598-4139-a360-6505ffe6dfb5 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:53.847185 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1421428-7caf-4d13-a51d-9312cef71810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:53.847800 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1421428-7caf-4d13-a51d-9312cef71810 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:53.848189 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1421428-7caf-4d13-a51d-9312cef71810 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:53.848478 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1421428-7caf-4d13-a51d-9312cef71810 None admin] PATH_INFO: `/v3/services` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:53.849191 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1421428-7caf-4d13-a51d-9312cef71810 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:53.849498 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1421428-7caf-4d13-a51d-9312cef71810 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:53.850876 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1421428-7caf-4d13-a51d-9312cef71810 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:53.855592 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 495/998] 10.222.0.26 () {56 vars in 1112 bytes} [Thu Jun 23 01:44:53 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:53.856344 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1421428-7caf-4d13-a51d-9312cef71810 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:53.862485 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a24da6-6224-4397-8f2e-6f2980001421 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:53.896665 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6524e59f-855b-42d0-b04a-a587da03ba1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:53.897442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6524e59f-855b-42d0-b04a-a587da03ba1b None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:53.897723 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6524e59f-855b-42d0-b04a-a587da03ba1b None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:53.897996 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6524e59f-855b-42d0-b04a-a587da03ba1b None admin] PATH_INFO: `/v3/services/image` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:53.898654 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6524e59f-855b-42d0-b04a-a587da03ba1b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:53.900057 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6524e59f-855b-42d0-b04a-a587da03ba1b None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:53.911717 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-6524e59f-855b-42d0-b04a-a587da03ba1b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 01:44:53.911717 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:53.911717 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 01:44:53.911717 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 01:44:53.911717 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 01:44:53.911717 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 01:44:53.911717 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 01:44:53.911717 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:44:53.911717 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 01:44:53.911717 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:53.911717 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 01:44:53.911717 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:53.911717 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:53.911717 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:53.911717 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:53.911717 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:53.911717 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:53.912977 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:53.912977 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:53.912977 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:53.912977 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:53.912977 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:53.912977 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:53.912977 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 01:44:53.912977 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 01:44:53.912977 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 01:44:53.912977 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 01:44:53.912977 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:53.912977 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:53.912977 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:53.912977 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:53.912977 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:53.912977 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:53.913812 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:53.913812 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:53.913812 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:53.913812 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:53.913812 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:53.913812 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:53.913812 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:53.913812 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:53.913812 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:53.913812 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:53.913812 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 01:44:53.913812 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:44:53.913812 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 01:44:53.913812 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:53.913812 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 504/999] 10.222.0.26 () {56 vars in 1105 bytes} [Thu Jun 23 01:44:53 2022] GET /identity/v3/services/image => generated 86 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:53.914918 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6524e59f-855b-42d0-b04a-a587da03ba1b None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:53.926472 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:53.948657 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:53.960293 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1421428-7caf-4d13-a51d-9312cef71810 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:53.993289 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0f36a-466c-4e0c-b438-210dc2208dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:53.993983 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0f36a-466c-4e0c-b438-210dc2208dbd None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:53.994264 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0f36a-466c-4e0c-b438-210dc2208dbd None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:53.994519 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0f36a-466c-4e0c-b438-210dc2208dbd None admin] PATH_INFO: `/v3/services` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:53.995270 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb0f36a-466c-4e0c-b438-210dc2208dbd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:53.995602 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb0f36a-466c-4e0c-b438-210dc2208dbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:53.996957 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb0f36a-466c-4e0c-b438-210dc2208dbd None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:54.007145 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 496/1000] 10.222.0.26 () {56 vars in 1122 bytes} [Thu Jun 23 01:44:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:54.007145 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0f36a-466c-4e0c-b438-210dc2208dbd None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:54.018820 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6524e59f-855b-42d0-b04a-a587da03ba1b None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:54.054012 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e414ca9-fc0c-4992-bd85-8ba134397723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:54.054995 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e414ca9-fc0c-4992-bd85-8ba134397723 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:54.055307 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e414ca9-fc0c-4992-bd85-8ba134397723 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:54.055577 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e414ca9-fc0c-4992-bd85-8ba134397723 None admin] PATH_INFO: `/v3/services` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:54.056288 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e414ca9-fc0c-4992-bd85-8ba134397723 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:54.056588 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e414ca9-fc0c-4992-bd85-8ba134397723 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:54.057963 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e414ca9-fc0c-4992-bd85-8ba134397723 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:54.062837 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 505/1001] 10.222.0.26 () {56 vars in 1112 bytes} [Thu Jun 23 01:44:53 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:54.063645 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e414ca9-fc0c-4992-bd85-8ba134397723 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:54.104745 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0f36a-466c-4e0c-b438-210dc2208dbd None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:54.133007 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:54.138500 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604fd82d-0b3b-4ce9-8771-ab5e986f43c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:54.139399 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604fd82d-0b3b-4ce9-8771-ab5e986f43c3 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:54.139539 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604fd82d-0b3b-4ce9-8771-ab5e986f43c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:54.139726 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604fd82d-0b3b-4ce9-8771-ab5e986f43c3 None admin] PATH_INFO: `/v3/services` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:54.140433 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604fd82d-0b3b-4ce9-8771-ab5e986f43c3 None admin] RBAC: Adding query filter params (name=image) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:54.140738 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604fd82d-0b3b-4ce9-8771-ab5e986f43c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:54.142111 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604fd82d-0b3b-4ce9-8771-ab5e986f43c3 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:54.147126 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 497/1002] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:54 2022] GET /identity/v3/services?name=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:54.147922 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604fd82d-0b3b-4ce9-8771-ab5e986f43c3 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:54.154762 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:54.161340 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e414ca9-fc0c-4992-bd85-8ba134397723 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:54.195298 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9c4cd2-9b60-4edc-a956-ad34b3b974db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:54.195970 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c4cd2-9b60-4edc-a956-ad34b3b974db None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:54.196253 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c4cd2-9b60-4edc-a956-ad34b3b974db None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:54.196525 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c4cd2-9b60-4edc-a956-ad34b3b974db None admin] PATH_INFO: `/v3/services` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:54.197374 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9c4cd2-9b60-4edc-a956-ad34b3b974db None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:54.197785 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9c4cd2-9b60-4edc-a956-ad34b3b974db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:54.199193 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9c4cd2-9b60-4edc-a956-ad34b3b974db None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:54.204805 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 506/1003] 10.222.0.26 () {56 vars in 1122 bytes} [Thu Jun 23 01:44:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:54.205659 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9c4cd2-9b60-4edc-a956-ad34b3b974db None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:54.257426 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604fd82d-0b3b-4ce9-8771-ab5e986f43c3 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:54.294563 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407737b-0892-4a78-9d81-bde12d66279f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:54.294907 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407737b-0892-4a78-9d81-bde12d66279f None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:54.295190 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407737b-0892-4a78-9d81-bde12d66279f None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:54.295494 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407737b-0892-4a78-9d81-bde12d66279f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:54.296298 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1407737b-0892-4a78-9d81-bde12d66279f None admin] RBAC: Adding query filter params (interface=public, service_id=b54da86a538842338a1826f649486b8e, region_id=RegionOne) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:54.296612 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1407737b-0892-4a78-9d81-bde12d66279f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:54.297987 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1407737b-0892-4a78-9d81-bde12d66279f None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:54.303134 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 498/1004] 10.222.0.26 () {56 vars in 1251 bytes} [Thu Jun 23 01:44:54 2022] GET /identity/v3/endpoints?service_id=b54da86a538842338a1826f649486b8e&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:54.303914 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407737b-0892-4a78-9d81-bde12d66279f None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:54.307977 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9c4cd2-9b60-4edc-a956-ad34b3b974db None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:54.340593 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd654e7c-ec31-4bb3-abcf-41e644a68307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:54.341245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd654e7c-ec31-4bb3-abcf-41e644a68307 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:54.341515 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd654e7c-ec31-4bb3-abcf-41e644a68307 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:54.341784 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd654e7c-ec31-4bb3-abcf-41e644a68307 None admin] PATH_INFO: `/v3/services` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:54.342478 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd654e7c-ec31-4bb3-abcf-41e644a68307 None admin] RBAC: Adding query filter params (type=image) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:54.342776 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd654e7c-ec31-4bb3-abcf-41e644a68307 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:54.344169 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd654e7c-ec31-4bb3-abcf-41e644a68307 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:54.348507 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 507/1005] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:54 2022] GET /identity/v3/services?type=image => generated 122 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:54.349252 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd654e7c-ec31-4bb3-abcf-41e644a68307 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:54.364654 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:54.391185 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:54.413596 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407737b-0892-4a78-9d81-bde12d66279f None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:54.453596 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0209dc02-4591-4ba1-97d6-ecbcad891d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:54.454649 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209dc02-4591-4ba1-97d6-ecbcad891d60 None admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:54.454975 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209dc02-4591-4ba1-97d6-ecbcad891d60 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:54.455272 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209dc02-4591-4ba1-97d6-ecbcad891d60 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:54.456379 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0209dc02-4591-4ba1-97d6-ecbcad891d60 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:54.458017 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0209dc02-4591-4ba1-97d6-ecbcad891d60 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:54.475480 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 499/1006] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Jun 23 01:44:54 2022] POST /identity/v3/endpoints => generated 325 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:44:54.481023 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd654e7c-ec31-4bb3-abcf-41e644a68307 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:54.529044 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6a0dbc-5c9f-41a9-ad45-fb4da81c14b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:54.530002 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a0dbc-5c9f-41a9-ad45-fb4da81c14b1 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:54.530282 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a0dbc-5c9f-41a9-ad45-fb4da81c14b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:54.530553 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a0dbc-5c9f-41a9-ad45-fb4da81c14b1 None admin] PATH_INFO: `/v3/services` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:54.531298 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6a0dbc-5c9f-41a9-ad45-fb4da81c14b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:54.532745 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6a0dbc-5c9f-41a9-ad45-fb4da81c14b1 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:54.540254 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 508/1007] 10.222.0.26 () {56 vars in 1087 bytes} [Thu Jun 23 01:44:54 2022] GET /identity/v3/services => generated 1727 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:44:56.348815 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4656ede6-a560-4565-a66d-14aa5fe0dd82 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:56.349349 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4656ede6-a560-4565-a66d-14aa5fe0dd82 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:56.349349 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4656ede6-a560-4565-a66d-14aa5fe0dd82 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:56.351241 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 500/1008] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:56.356187 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452301b9-7ff8-4ac6-87e6-6fa7b66f5bcc None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:56.356607 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452301b9-7ff8-4ac6-87e6-6fa7b66f5bcc None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:56.356768 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452301b9-7ff8-4ac6-87e6-6fa7b66f5bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:56.374357 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-452301b9-7ff8-4ac6-87e6-6fa7b66f5bcc None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:56.375592 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-452301b9-7ff8-4ac6-87e6-6fa7b66f5bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:56.448916 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 509/1009] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:56 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:44:56.471315 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4656ede6-a560-4565-a66d-14aa5fe0dd82 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:56.498132 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c6fd6-8239-4c4d-ab33-ae4dee190039 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:56.498459 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c6fd6-8239-4c4d-ab33-ae4dee190039 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:56.498656 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c6fd6-8239-4c4d-ab33-ae4dee190039 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:56.500077 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 510/1010] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:56.505460 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adcbd6e-2411-41e5-bc5a-c3716117eb06 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:56.505745 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adcbd6e-2411-41e5-bc5a-c3716117eb06 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:56.506020 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adcbd6e-2411-41e5-bc5a-c3716117eb06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:56.524669 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-9adcbd6e-2411-41e5-bc5a-c3716117eb06 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:56.525859 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-9adcbd6e-2411-41e5-bc5a-c3716117eb06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:56.536534 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:56.579403 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4656ede6-a560-4565-a66d-14aa5fe0dd82 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:56.595932 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 511/1011] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:56 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:44:56.597419 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95093c3-2478-42e7-82c4-1b6ee2ce58f7 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:56.597705 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95093c3-2478-42e7-82c4-1b6ee2ce58f7 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:56.597975 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95093c3-2478-42e7-82c4-1b6ee2ce58f7 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:56.599277 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:56.603609 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 512/1012] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:56.604769 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3188db3-2a38-45db-aae5-322f78103176 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:56.605027 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3188db3-2a38-45db-aae5-322f78103176 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:56.605302 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3188db3-2a38-45db-aae5-322f78103176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:56.616716 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813dd56f-ec15-4384-be6f-62c9aa51ee93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:56.617382 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813dd56f-ec15-4384-be6f-62c9aa51ee93 None admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:56.617659 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813dd56f-ec15-4384-be6f-62c9aa51ee93 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:56.617931 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813dd56f-ec15-4384-be6f-62c9aa51ee93 None admin] PATH_INFO: `/v3/services` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:56.618725 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813dd56f-ec15-4384-be6f-62c9aa51ee93 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:56.620129 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813dd56f-ec15-4384-be6f-62c9aa51ee93 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:56.622131 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-d3188db3-2a38-45db-aae5-322f78103176 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:56.623404 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-d3188db3-2a38-45db-aae5-322f78103176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:56.635368 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 501/1013] 10.222.0.26 () {60 vars in 1141 bytes} [Thu Jun 23 01:44:56 2022] POST /identity/v3/services => generated 242 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:44:56.637279 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813dd56f-ec15-4384-be6f-62c9aa51ee93 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:56.712279 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 513/1014] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:56 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:44:56.734971 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3188db3-2a38-45db-aae5-322f78103176 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:56.753510 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813dd56f-ec15-4384-be6f-62c9aa51ee93 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:56.790333 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6301a7-dbff-4749-ac0a-61a591afb400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:56.791578 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6301a7-dbff-4749-ac0a-61a591afb400 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:56.791875 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6301a7-dbff-4749-ac0a-61a591afb400 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:56.792158 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6301a7-dbff-4749-ac0a-61a591afb400 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:56.793348 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6301a7-dbff-4749-ac0a-61a591afb400 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:56.794780 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6301a7-dbff-4749-ac0a-61a591afb400 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:56.796732 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 502/1015] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:56.802383 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6301a7-dbff-4749-ac0a-61a591afb400 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:56.810017 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:56.847463 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3188db3-2a38-45db-aae5-322f78103176 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:56.858756 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:56.882443 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de6f21c-5a2d-4d5e-9795-2e3183b0e2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:56.883505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de6f21c-5a2d-4d5e-9795-2e3183b0e2b2 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:56.883505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de6f21c-5a2d-4d5e-9795-2e3183b0e2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:56.883828 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de6f21c-5a2d-4d5e-9795-2e3183b0e2b2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:56.884442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de6f21c-5a2d-4d5e-9795-2e3183b0e2b2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:56.885822 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de6f21c-5a2d-4d5e-9795-2e3183b0e2b2 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:56.897752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-7de6f21c-5a2d-4d5e-9795-2e3183b0e2b2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 01:44:56.897752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:56.897752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 01:44:56.897752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 01:44:56.897752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 01:44:56.897752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 01:44:56.897752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 01:44:56.897752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:44:56.897752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 01:44:56.897752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:56.897752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 01:44:56.897752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:56.897752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:56.897752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:56.897752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:56.897752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:56.897752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:56.898844 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:56.898844 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:56.898844 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:56.898844 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:56.898844 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:56.898844 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:56.898844 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 01:44:56.898844 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 01:44:56.898844 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 01:44:56.898844 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 01:44:56.898844 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:56.898844 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:56.898844 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:56.898844 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:56.898844 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:56.898844 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:56.899876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:56.899876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:56.899876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:56.899876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:56.899876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:56.899876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:56.899876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:56.899876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:56.899876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:56.899876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:56.899876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 01:44:56.899876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:44:56.899876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 01:44:56.899876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:56.899876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 514/1016] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:44:56 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:44:56.904661 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de6f21c-5a2d-4d5e-9795-2e3183b0e2b2 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:56.917927 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6301a7-dbff-4749-ac0a-61a591afb400 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:56.954801 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b8cdbc-c0bf-43bc-b5a6-1ff4d6376bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:56.956351 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8cdbc-c0bf-43bc-b5a6-1ff4d6376bc9 None admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:56.957001 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8cdbc-c0bf-43bc-b5a6-1ff4d6376bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:56.957522 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8cdbc-c0bf-43bc-b5a6-1ff4d6376bc9 None admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:56.958522 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b8cdbc-c0bf-43bc-b5a6-1ff4d6376bc9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:56.960322 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b8cdbc-c0bf-43bc-b5a6-1ff4d6376bc9 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:56.989381 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 503/1017] 10.222.0.26 () {60 vars in 1141 bytes} [Thu Jun 23 01:44:56 2022] POST /identity/v3/projects => generated 311 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 01:44:57.015936 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de6f21c-5a2d-4d5e-9795-2e3183b0e2b2 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:57.053410 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f14224-0e67-4d88-b40c-8b7efe152e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:57.054082 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f14224-0e67-4d88-b40c-8b7efe152e94 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:57.054640 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f14224-0e67-4d88-b40c-8b7efe152e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:57.054967 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f14224-0e67-4d88-b40c-8b7efe152e94 None admin] PATH_INFO: `/v3/services` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:57.055692 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f14224-0e67-4d88-b40c-8b7efe152e94 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:57.055993 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f14224-0e67-4d88-b40c-8b7efe152e94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:57.057389 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f14224-0e67-4d88-b40c-8b7efe152e94 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:57.058718 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:57.068781 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 515/1018] 10.222.0.26 () {56 vars in 1112 bytes} [Thu Jun 23 01:44:56 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:44:57.070105 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b8cdbc-c0bf-43bc-b5a6-1ff4d6376bc9 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:57.176548 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:57.180214 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b8cdbc-c0bf-43bc-b5a6-1ff4d6376bc9 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:57.219032 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae4b85-d165-4f3d-8fb7-cbae006320e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:57.219756 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae4b85-d165-4f3d-8fb7-cbae006320e5 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:57.220038 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae4b85-d165-4f3d-8fb7-cbae006320e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:57.220333 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae4b85-d165-4f3d-8fb7-cbae006320e5 None admin] PATH_INFO: `/v3/services` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:57.221048 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ae4b85-d165-4f3d-8fb7-cbae006320e5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:57.221348 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ae4b85-d165-4f3d-8fb7-cbae006320e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:57.222713 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ae4b85-d165-4f3d-8fb7-cbae006320e5 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:57.227859 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 504/1019] 10.222.0.26 () {56 vars in 1112 bytes} [Thu Jun 23 01:44:57 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:57.233703 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f14224-0e67-4d88-b40c-8b7efe152e94 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:57.340793 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f14224-0e67-4d88-b40c-8b7efe152e94 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:57.374780 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c783b-4980-4b77-8546-4dc534bb4639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:57.375546 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c783b-4980-4b77-8546-4dc534bb4639 None admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:57.375735 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c783b-4980-4b77-8546-4dc534bb4639 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:57.376012 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c783b-4980-4b77-8546-4dc534bb4639 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:57.376815 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c783b-4980-4b77-8546-4dc534bb4639 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:57.378184 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c783b-4980-4b77-8546-4dc534bb4639 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:57.398437 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 516/1020] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Jun 23 01:44:57 2022] POST /identity/v3/endpoints => generated 334 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:44:58.664500 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fcad44-b4b3-4325-b7dd-dca885e99f7b None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:58.665008 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fcad44-b4b3-4325-b7dd-dca885e99f7b None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:58.665008 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fcad44-b4b3-4325-b7dd-dca885e99f7b None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:58.666292 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 505/1021] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:58.671690 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830a56c-b3b7-4d62-9c09-cc301c669901 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:58.672094 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830a56c-b3b7-4d62-9c09-cc301c669901 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:58.672388 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830a56c-b3b7-4d62-9c09-cc301c669901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:58.690457 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-0830a56c-b3b7-4d62-9c09-cc301c669901 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:58.691678 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-0830a56c-b3b7-4d62-9c09-cc301c669901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:58.744472 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:58.769982 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 517/1022] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:58 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:44:58.792147 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fcad44-b4b3-4325-b7dd-dca885e99f7b None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:58.900375 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fcad44-b4b3-4325-b7dd-dca885e99f7b None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:58.936482 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb80e2a-a46a-42f5-b7c0-493d98b56bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:58.937282 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb80e2a-a46a-42f5-b7c0-493d98b56bce None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:58.937563 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb80e2a-a46a-42f5-b7c0-493d98b56bce None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:58.937835 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb80e2a-a46a-42f5-b7c0-493d98b56bce None admin] PATH_INFO: `/v3/services/image` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:58.938493 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb80e2a-a46a-42f5-b7c0-493d98b56bce None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:58.939946 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb80e2a-a46a-42f5-b7c0-493d98b56bce None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:58.951624 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-5bb80e2a-a46a-42f5-b7c0-493d98b56bce None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 01:44:58.951624 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:58.951624 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 01:44:58.951624 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 01:44:58.951624 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 01:44:58.951624 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 01:44:58.951624 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 01:44:58.951624 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:44:58.951624 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 01:44:58.951624 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:58.951624 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 01:44:58.951624 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:58.951624 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:58.951624 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:58.951624 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:58.951624 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:58.951624 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:58.952600 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:58.952600 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:58.952600 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:58.952600 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:58.952600 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:58.952600 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:58.952600 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 01:44:58.952600 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 01:44:58.952600 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 01:44:58.952600 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 01:44:58.952600 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:58.952600 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:58.952600 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:58.952600 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:58.952600 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:58.952600 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:58.958462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:58.958462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:58.958462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:58.958462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:58.958462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:58.958462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:58.958462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:58.958462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:58.958462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:58.958462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:58.958462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 01:44:58.958462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:44:58.958462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 01:44:58.958462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:44:58.958462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 506/1023] 10.222.0.26 () {56 vars in 1105 bytes} [Thu Jun 23 01:44:58 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:58.959265 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0830a56c-b3b7-4d62-9c09-cc301c669901 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:59.060849 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0830a56c-b3b7-4d62-9c09-cc301c669901 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:59.071584 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1cb5d6-786f-4048-b0cc-cf355619300f None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:59.071848 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1cb5d6-786f-4048-b0cc-cf355619300f None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:59.072394 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1cb5d6-786f-4048-b0cc-cf355619300f None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:59.073747 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 507/1024] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 01:44:59.078732 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf7357-7101-43e6-97ed-b1a294477053 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:59.079243 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf7357-7101-43e6-97ed-b1a294477053 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:59.079656 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf7357-7101-43e6-97ed-b1a294477053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:59.096409 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e826fc-0bd2-4b66-a41a-83915798883b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:59.096894 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-fcbf7357-7101-43e6-97ed-b1a294477053 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:59.097084 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e826fc-0bd2-4b66-a41a-83915798883b None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:59.097348 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e826fc-0bd2-4b66-a41a-83915798883b None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:59.097616 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e826fc-0bd2-4b66-a41a-83915798883b None admin] PATH_INFO: `/v3/services` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:59.098103 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-fcbf7357-7101-43e6-97ed-b1a294477053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:59.098331 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e826fc-0bd2-4b66-a41a-83915798883b None admin] RBAC: Adding query filter params (name=image) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:59.098631 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e826fc-0bd2-4b66-a41a-83915798883b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:59.100112 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e826fc-0bd2-4b66-a41a-83915798883b None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:59.105171 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 518/1025] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:58 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:44:59.110534 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e826fc-0bd2-4b66-a41a-83915798883b None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:59.123223 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:59.160314 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:59.175267 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 508/1026] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:59 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:44:59.198167 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbf7357-7101-43e6-97ed-b1a294477053 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:59.216977 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e826fc-0bd2-4b66-a41a-83915798883b None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:59.255560 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e67ddef-8251-4627-ab40-4dbd0435e5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:59.256221 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67ddef-8251-4627-ab40-4dbd0435e5ff None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:59.256509 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67ddef-8251-4627-ab40-4dbd0435e5ff None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:59.256780 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67ddef-8251-4627-ab40-4dbd0435e5ff None admin] PATH_INFO: `/v3/services` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:59.257492 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e67ddef-8251-4627-ab40-4dbd0435e5ff None admin] RBAC: Adding query filter params (type=image) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:59.257784 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e67ddef-8251-4627-ab40-4dbd0435e5ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:59.259163 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e67ddef-8251-4627-ab40-4dbd0435e5ff None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:59.272120 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 519/1027] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:59 2022] GET /identity/v3/services?type=image => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:44:59.273707 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e67ddef-8251-4627-ab40-4dbd0435e5ff None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:59.310720 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbf7357-7101-43e6-97ed-b1a294477053 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:59.347730 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94be259d-5580-458d-8bf5-0a2e3789308b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:59.348535 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be259d-5580-458d-8bf5-0a2e3789308b None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:59.348816 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be259d-5580-458d-8bf5-0a2e3789308b None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:59.349090 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be259d-5580-458d-8bf5-0a2e3789308b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:59.350302 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94be259d-5580-458d-8bf5-0a2e3789308b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:59.351861 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94be259d-5580-458d-8bf5-0a2e3789308b None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:59.358138 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 509/1028] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:44:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:44:59.359558 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94be259d-5580-458d-8bf5-0a2e3789308b None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:59.378007 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e67ddef-8251-4627-ab40-4dbd0435e5ff None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:59.387672 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:59.392654 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:59.413120 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f9622-0f7b-4ecf-a773-58b0530c6c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:59.413780 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f9622-0f7b-4ecf-a773-58b0530c6c04 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:59.414062 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f9622-0f7b-4ecf-a773-58b0530c6c04 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:59.414341 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f9622-0f7b-4ecf-a773-58b0530c6c04 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:59.415131 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26f9622-0f7b-4ecf-a773-58b0530c6c04 None admin] RBAC: Adding query filter params (interface=public, service_id=a36a0d49cde4494ca79b8ace747e64f0, region_id=RegionOne) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:59.415472 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26f9622-0f7b-4ecf-a773-58b0530c6c04 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:59.416856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26f9622-0f7b-4ecf-a773-58b0530c6c04 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:59.421895 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 520/1029] 10.222.0.26 () {56 vars in 1251 bytes} [Thu Jun 23 01:44:59 2022] GET /identity/v3/endpoints?service_id=a36a0d49cde4494ca79b8ace747e64f0&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:44:59.427658 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f9622-0f7b-4ecf-a773-58b0530c6c04 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:59.475807 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94be259d-5580-458d-8bf5-0a2e3789308b None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:59.510496 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d3bdf2-3c21-42f7-981e-06bf1d10cabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:59.511177 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3bdf2-3c21-42f7-981e-06bf1d10cabe None admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:59.511485 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3bdf2-3c21-42f7-981e-06bf1d10cabe None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:59.511757 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3bdf2-3c21-42f7-981e-06bf1d10cabe None admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:59.512468 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d3bdf2-3c21-42f7-981e-06bf1d10cabe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:59.513890 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d3bdf2-3c21-42f7-981e-06bf1d10cabe None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:59.533399 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f9622-0f7b-4ecf-a773-58b0530c6c04 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:59.534312 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 510/1030] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Jun 23 01:44:59 2022] POST /identity/v3/users => generated 285 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 01:44:59.535675 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b882162-f194-493a-9e74-1b7096221db7 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:59.536024 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b882162-f194-493a-9e74-1b7096221db7 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:59.536362 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b882162-f194-493a-9e74-1b7096221db7 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:59.537690 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 511/1031] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:44:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:44:59.543355 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb20f24e-11d6-4d31-82dd-aa4ee3581dfc None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:59.543644 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb20f24e-11d6-4d31-82dd-aa4ee3581dfc None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:59.543921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb20f24e-11d6-4d31-82dd-aa4ee3581dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:59.562117 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-bb20f24e-11d6-4d31-82dd-aa4ee3581dfc None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:44:59.563379 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-bb20f24e-11d6-4d31-82dd-aa4ee3581dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:44:59.591050 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817fe7a5-d536-47c5-9122-2f1feeba493b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:59.591763 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817fe7a5-d536-47c5-9122-2f1feeba493b None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:59.592047 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817fe7a5-d536-47c5-9122-2f1feeba493b None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:59.592322 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817fe7a5-d536-47c5-9122-2f1feeba493b None admin] PATH_INFO: `/v3/services` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:59.593028 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817fe7a5-d536-47c5-9122-2f1feeba493b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:44:59.594463 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817fe7a5-d536-47c5-9122-2f1feeba493b None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:44:59.602322 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 521/1032] 10.222.0.26 () {56 vars in 1087 bytes} [Thu Jun 23 01:44:59 2022] GET /identity/v3/services => generated 1957 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:44:59.651592 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 512/1033] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:44:59 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 01:44:59.676264 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817fe7a5-d536-47c5-9122-2f1feeba493b None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:59.717432 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:59.790722 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817fe7a5-d536-47c5-9122-2f1feeba493b None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:59.825513 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1306c2-ecb1-4d32-a8c0-acc3a295bfba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:59.826325 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1306c2-ecb1-4d32-a8c0-acc3a295bfba None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:59.826667 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1306c2-ecb1-4d32-a8c0-acc3a295bfba None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:59.826937 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1306c2-ecb1-4d32-a8c0-acc3a295bfba None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:59.837349 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1306c2-ecb1-4d32-a8c0-acc3a295bfba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:44:59.845549 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-8e1306c2-ecb1-4d32-a8c0-acc3a295bfba None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 01:44:59.845549 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:44:59.845549 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:44:59.845549 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:44:59.845549 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:44:59.845549 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:44:59.845549 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:44:59.845549 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:44:59.845549 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:44:59.845549 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:44:59.845549 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:44:59.845549 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:44:59.845549 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:44:59.845549 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:44:59.845549 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:44:59.845549 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:44:59.846457 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:44:59.846457 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:44:59.846457 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:44:59.846457 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:44:59.846457 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:44:59.846457 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:44:59.846457 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:44:59.846457 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:44:59.846457 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:44:59.846457 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:44:59.846457 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:44:59.846457 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:44:59.846457 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:44:59.846457 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:44:59.846457 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:44:59.846457 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:44:59.847311 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:44:59.847311 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:44:59.847311 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:44:59.847311 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:44:59.847311 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:44:59.847311 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:44:59.847311 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:44:59.847311 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:44:59.847311 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:44:59.847311 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:44:59.847311 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 01:44:59.847311 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:44:59.847311 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 522/1034] 10.222.0.26 () {56 vars in 1120 bytes} [Thu Jun 23 01:44:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:44:59.856714 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb20f24e-11d6-4d31-82dd-aa4ee3581dfc None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:44:59.871612 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:44:59.962417 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb20f24e-11d6-4d31-82dd-aa4ee3581dfc None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:44:59.997034 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec51b452-1fb0-46b0-8c40-3febd92b1850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:44:59.997709 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51b452-1fb0-46b0-8c40-3febd92b1850 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:44:59.997987 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51b452-1fb0-46b0-8c40-3febd92b1850 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:44:59.998262 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51b452-1fb0-46b0-8c40-3febd92b1850 None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:44:59.999000 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec51b452-1fb0-46b0-8c40-3febd92b1850 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:44:59.999343 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec51b452-1fb0-46b0-8c40-3febd92b1850 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:00.000720 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec51b452-1fb0-46b0-8c40-3febd92b1850 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:00.009761 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 513/1035] 10.222.0.26 () {56 vars in 1115 bytes} [Thu Jun 23 01:44:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:00.015263 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1306c2-ecb1-4d32-a8c0-acc3a295bfba None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:00.116387 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:00.123483 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1306c2-ecb1-4d32-a8c0-acc3a295bfba None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:00.157432 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f1a685-1673-44e2-8857-1961196b3d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:00.158600 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1a685-1673-44e2-8857-1961196b3d8b None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:00.159098 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1a685-1673-44e2-8857-1961196b3d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:00.159570 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1a685-1673-44e2-8857-1961196b3d8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:00.171205 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f1a685-1673-44e2-8857-1961196b3d8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:00.172897 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f1a685-1673-44e2-8857-1961196b3d8b None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:00.190728 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-52f1a685-1673-44e2-8857-1961196b3d8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:45:00.190728 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:45:00.190728 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:45:00.190728 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:45:00.190728 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:45:00.190728 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:45:00.190728 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:45:00.190728 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:45:00.190728 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:45:00.190728 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:45:00.190728 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:45:00.190728 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:45:00.190728 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:45:00.190728 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:45:00.190728 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:45:00.191745 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:45:00.191745 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:45:00.191745 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:45:00.191745 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:45:00.191745 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:45:00.191745 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:45:00.191745 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:45:00.191745 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:45:00.191745 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:45:00.191745 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:45:00.191745 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:45:00.191745 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:45:00.191745 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:45:00.191745 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:45:00.191745 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:45:00.191745 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:45:00.191745 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:45:00.197566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:45:00.197566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:45:00.197566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:45:00.197566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:45:00.197566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 523/1036] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:45:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:45:00.198779 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec51b452-1fb0-46b0-8c40-3febd92b1850 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:00.279777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:00.312232 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec51b452-1fb0-46b0-8c40-3febd92b1850 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:00.345717 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d88132-1157-4b5d-ae72-809d9642fa9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:00.346383 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d88132-1157-4b5d-ae72-809d9642fa9a None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:00.346656 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d88132-1157-4b5d-ae72-809d9642fa9a None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:00.346927 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d88132-1157-4b5d-ae72-809d9642fa9a None admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:00.347683 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d88132-1157-4b5d-ae72-809d9642fa9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:45:00.347985 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d88132-1157-4b5d-ae72-809d9642fa9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:00.349347 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d88132-1157-4b5d-ae72-809d9642fa9a None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:00.361596 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 514/1037] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:45:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:00.367124 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f1a685-1673-44e2-8857-1961196b3d8b None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:00.467465 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f1a685-1673-44e2-8857-1961196b3d8b None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:00.507257 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0448cdee-c37d-4807-8fb8-124af9c2b8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:00.508061 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0448cdee-c37d-4807-8fb8-124af9c2b8a6 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:00.508345 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0448cdee-c37d-4807-8fb8-124af9c2b8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:00.508618 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0448cdee-c37d-4807-8fb8-124af9c2b8a6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:00.516380 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0448cdee-c37d-4807-8fb8-124af9c2b8a6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:00.517858 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0448cdee-c37d-4807-8fb8-124af9c2b8a6 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:00.531760 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-0448cdee-c37d-4807-8fb8-124af9c2b8a6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 01:45:00.531760 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:45:00.531760 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:45:00.531760 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:45:00.531760 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:45:00.531760 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:45:00.531760 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:45:00.531760 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:45:00.531760 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:45:00.531760 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:45:00.531760 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:45:00.531760 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:45:00.531760 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 01:45:00.531760 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 01:45:00.531760 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 01:45:00.531760 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 01:45:00.532720 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:45:00.532720 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:45:00.532720 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 01:45:00.532720 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:45:00.532720 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 01:45:00.532720 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:45:00.532720 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:45:00.532720 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:45:00.532720 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:45:00.532720 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:45:00.532720 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:45:00.532720 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:45:00.532720 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:45:00.532720 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:45:00.532720 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:45:00.532720 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:45:00.532720 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:45:00.532720 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:45:00.533677 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:45:00.533677 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:45:00.533677 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 01:45:00.533677 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 01:45:00.533677 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 01:45:00.533677 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 01:45:00.533677 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 01:45:00.533677 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 01:45:00.533677 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 01:45:00.533677 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:45:00.533677 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 524/1038] 10.222.0.26 () {56 vars in 1093 bytes} [Thu Jun 23 01:45:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:45:00.540333 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d88132-1157-4b5d-ae72-809d9642fa9a None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:00.642647 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d88132-1157-4b5d-ae72-809d9642fa9a None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:00.676919 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81af00c4-7507-4df3-aa39-8ceefbefc05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:00.677592 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af00c4-7507-4df3-aa39-8ceefbefc05a None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:00.677871 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af00c4-7507-4df3-aa39-8ceefbefc05a None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:00.678142 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af00c4-7507-4df3-aa39-8ceefbefc05a None admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:00.678979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81af00c4-7507-4df3-aa39-8ceefbefc05a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:45:00.679627 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81af00c4-7507-4df3-aa39-8ceefbefc05a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:00.681082 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81af00c4-7507-4df3-aa39-8ceefbefc05a None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:00.682838 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:00.703308 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 515/1039] 10.222.0.26 () {56 vars in 1133 bytes} [Thu Jun 23 01:45:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:00.709208 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0448cdee-c37d-4807-8fb8-124af9c2b8a6 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:00.721309 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:00.814943 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0448cdee-c37d-4807-8fb8-124af9c2b8a6 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:00.848650 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def89f11-0a4d-4357-ac73-970babb35ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:00.849287 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def89f11-0a4d-4357-ac73-970babb35ec5 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:00.849563 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def89f11-0a4d-4357-ac73-970babb35ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:00.849831 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def89f11-0a4d-4357-ac73-970babb35ec5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:00.914608 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def89f11-0a4d-4357-ac73-970babb35ec5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:00.916011 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def89f11-0a4d-4357-ac73-970babb35ec5 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:00.925318 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:01.012269 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 525/1040] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:45:00 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 01:45:01.018469 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81af00c4-7507-4df3-aa39-8ceefbefc05a None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:01.121597 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81af00c4-7507-4df3-aa39-8ceefbefc05a None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:01.156656 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550fc78a-6dc7-43c3-a16e-ecb0440de1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:01.157450 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550fc78a-6dc7-43c3-a16e-ecb0440de1d5 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:01.157791 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550fc78a-6dc7-43c3-a16e-ecb0440de1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:01.158063 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550fc78a-6dc7-43c3-a16e-ecb0440de1d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:01.168558 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550fc78a-6dc7-43c3-a16e-ecb0440de1d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:01.169940 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550fc78a-6dc7-43c3-a16e-ecb0440de1d5 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:01.188147 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-550fc78a-6dc7-43c3-a16e-ecb0440de1d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:45:01.188147 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:45:01.188147 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:45:01.188147 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:45:01.188147 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:45:01.188147 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:45:01.188147 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:45:01.188147 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:45:01.188147 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:45:01.188147 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:45:01.188147 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:45:01.188147 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:45:01.188147 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:45:01.188147 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:45:01.188147 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:45:01.189217 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:45:01.189217 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:45:01.189217 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:45:01.189217 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:45:01.189217 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:45:01.189217 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:45:01.189217 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:45:01.189217 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:45:01.189217 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:45:01.189217 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:45:01.189217 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:45:01.189217 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:45:01.189217 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:45:01.189217 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:45:01.189217 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:45:01.189217 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:45:01.189217 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:45:01.190190 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:45:01.190190 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:45:01.190190 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:45:01.190190 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:45:01.191549 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 516/1041] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:45:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:45:01.195130 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def89f11-0a4d-4357-ac73-970babb35ec5 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:01.303050 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def89f11-0a4d-4357-ac73-970babb35ec5 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:01.313815 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:01.340730 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ce120e-8674-4846-a114-a06360c1054b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:01.341397 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce120e-8674-4846-a114-a06360c1054b None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:01.341680 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce120e-8674-4846-a114-a06360c1054b None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:01.341955 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce120e-8674-4846-a114-a06360c1054b None admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:01.342684 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ce120e-8674-4846-a114-a06360c1054b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:45:01.342989 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ce120e-8674-4846-a114-a06360c1054b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:01.344413 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ce120e-8674-4846-a114-a06360c1054b None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:01.356434 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 526/1042] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:45:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:01.361951 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550fc78a-6dc7-43c3-a16e-ecb0440de1d5 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:01.410465 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:01.466288 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550fc78a-6dc7-43c3-a16e-ecb0440de1d5 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:01.501272 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c19533-2c23-45a2-8e26-f49e58cabd5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:01.502119 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c19533-2c23-45a2-8e26-f49e58cabd5d None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:01.502405 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c19533-2c23-45a2-8e26-f49e58cabd5d None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:01.502683 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c19533-2c23-45a2-8e26-f49e58cabd5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:01.509756 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c19533-2c23-45a2-8e26-f49e58cabd5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:01.511236 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c19533-2c23-45a2-8e26-f49e58cabd5d None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:01.525427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-b7c19533-2c23-45a2-8e26-f49e58cabd5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:45:01.525427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:45:01.525427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:45:01.525427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:45:01.525427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:45:01.525427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:45:01.525427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:45:01.525427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:45:01.525427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:45:01.525427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:45:01.525427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:45:01.525427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:45:01.525427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 01:45:01.525427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 01:45:01.525427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 01:45:01.526339 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 01:45:01.526339 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:45:01.526339 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:45:01.526339 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:45:01.526339 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:45:01.526339 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:45:01.526339 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:45:01.526339 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:45:01.526339 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:45:01.526339 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:45:01.526339 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:45:01.526339 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:45:01.526339 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:45:01.526339 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:45:01.526339 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:45:01.526339 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:45:01.526339 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:45:01.527272 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 01:45:01.527272 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 01:45:01.527272 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 01:45:01.527272 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 01:45:01.527272 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 01:45:01.527272 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 01:45:01.527272 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:45:01.527272 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:45:01.531556 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 517/1043] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:45:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:45:01.533129 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ce120e-8674-4846-a114-a06360c1054b None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:01.602503 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be38c5-e2b6-4fda-963b-f694c2cbddc5 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:01.602896 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be38c5-e2b6-4fda-963b-f694c2cbddc5 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:01.603297 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be38c5-e2b6-4fda-963b-f694c2cbddc5 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:01.604796 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 518/1044] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:45:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:45:01.609734 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc368e7-a9ea-4f1f-8c87-0a2a102505ce None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:01.610024 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc368e7-a9ea-4f1f-8c87-0a2a102505ce None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:01.610300 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc368e7-a9ea-4f1f-8c87-0a2a102505ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:01.627285 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-dbc368e7-a9ea-4f1f-8c87-0a2a102505ce None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:45:01.628461 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-dbc368e7-a9ea-4f1f-8c87-0a2a102505ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:45:01.640684 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ce120e-8674-4846-a114-a06360c1054b None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:01.674118 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120458d5-4785-4933-83ae-67450e0a1d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:01.674794 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120458d5-4785-4933-83ae-67450e0a1d48 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:01.675074 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120458d5-4785-4933-83ae-67450e0a1d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:01.675341 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120458d5-4785-4933-83ae-67450e0a1d48 None admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:01.676123 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120458d5-4785-4933-83ae-67450e0a1d48 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:45:01.676424 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120458d5-4785-4933-83ae-67450e0a1d48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:01.677845 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120458d5-4785-4933-83ae-67450e0a1d48 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:01.695700 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 527/1045] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jun 23 01:45:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:45:01.695700 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc579b-446b-4027-adde-5b64826c690d None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:01.695700 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc579b-446b-4027-adde-5b64826c690d None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:01.695700 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc579b-446b-4027-adde-5b64826c690d None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:01.695700 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 528/1046] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:45:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:45:01.696641 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc579b-446b-4027-adde-5b64826c690d None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:01.707248 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 519/1047] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:45:01 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 01:45:01.707248 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e65f5-a9a5-4321-9b99-b412d90a6a92 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:01.707248 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e65f5-a9a5-4321-9b99-b412d90a6a92 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:01.707248 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e65f5-a9a5-4321-9b99-b412d90a6a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:01.716687 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-6a6e65f5-a9a5-4321-9b99-b412d90a6a92 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:45:01.717788 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-6a6e65f5-a9a5-4321-9b99-b412d90a6a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:45:01.724721 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:01.744037 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:01.783157 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 520/1048] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:45:01 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:45:01.784191 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e65f5-a9a5-4321-9b99-b412d90a6a92 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:01.816247 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc579b-446b-4027-adde-5b64826c690d None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:01.850083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88ac128-4335-4d95-a3d7-78621c7fee53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:01.850739 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ac128-4335-4d95-a3d7-78621c7fee53 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:01.851017 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ac128-4335-4d95-a3d7-78621c7fee53 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:01.851319 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ac128-4335-4d95-a3d7-78621c7fee53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:01.852163 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88ac128-4335-4d95-a3d7-78621c7fee53 None admin] RBAC: Adding query filter params (role.id=f173bf8d88d34d82881f578a6c10a7ce, scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c, user.id=ae4e83df33b64a3ebf28f54e2d94fe63) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:45:01.852475 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88ac128-4335-4d95-a3d7-78621c7fee53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:01.853925 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88ac128-4335-4d95-a3d7-78621c7fee53 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:01.859351 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 529/1049] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:45:01 2022] GET /identity/v3/role_assignments?user.id=ae4e83df33b64a3ebf28f54e2d94fe63&scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c&role.id=f173bf8d88d34d82881f578a6c10a7ce => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:45:01.860293 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88ac128-4335-4d95-a3d7-78621c7fee53 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:01.884911 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e65f5-a9a5-4321-9b99-b412d90a6a92 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:01.922270 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3c4505-567e-4b05-8693-0d78d7645105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:01.923082 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c4505-567e-4b05-8693-0d78d7645105 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:01.923436 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c4505-567e-4b05-8693-0d78d7645105 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:01.923708 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c4505-567e-4b05-8693-0d78d7645105 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:01.936023 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3c4505-567e-4b05-8693-0d78d7645105 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:45:01.944099 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-bf3c4505-567e-4b05-8693-0d78d7645105 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:45:01.944099 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:45:01.944099 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:45:01.944099 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:45:01.944099 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:45:01.944099 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:45:01.944099 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:45:01.944099 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:45:01.944099 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:45:01.944099 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:45:01.944099 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:45:01.944099 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:45:01.944099 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:45:01.944099 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:45:01.944099 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:45:01.944099 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:45:01.945136 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:45:01.945136 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:45:01.945136 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:45:01.945136 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:45:01.945136 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:45:01.945136 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:45:01.945136 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:45:01.945136 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:45:01.945136 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:45:01.945136 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:45:01.945136 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:45:01.945136 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:45:01.945136 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:45:01.945136 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:45:01.945136 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:45:01.945136 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:45:01.945982 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:45:01.945982 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:45:01.945982 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:45:01.945982 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:45:01.945982 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:45:01.945982 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:45:01.945982 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:45:01.945982 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:45:01.945982 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:45:01.945982 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:45:01.945982 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:45:01.945982 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:45:01.945982 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 521/1050] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jun 23 01:45:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:45:01.951629 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3c4505-567e-4b05-8693-0d78d7645105 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:01.969763 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:01.978472 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:01.982315 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88ac128-4335-4d95-a3d7-78621c7fee53 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:02.024722 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5012f3-6a3f-4ef8-8df6-9254e304db4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:02.025521 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5012f3-6a3f-4ef8-8df6-9254e304db4a None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:02.025801 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5012f3-6a3f-4ef8-8df6-9254e304db4a None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:02.026726 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5012f3-6a3f-4ef8-8df6-9254e304db4a None admin] PATH_INFO: `/v3/services/image` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:02.026726 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5012f3-6a3f-4ef8-8df6-9254e304db4a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:02.028267 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5012f3-6a3f-4ef8-8df6-9254e304db4a None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:02.042389 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-7e5012f3-6a3f-4ef8-8df6-9254e304db4a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 01:45:02.042389 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:45:02.042389 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 01:45:02.042389 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 01:45:02.042389 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 01:45:02.042389 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 01:45:02.042389 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 01:45:02.042389 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:45:02.042389 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 01:45:02.042389 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:45:02.042389 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 01:45:02.042389 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:45:02.042389 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:45:02.042389 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:45:02.042389 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:45:02.042389 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:45:02.042389 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:45:02.043497 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:45:02.043497 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:45:02.043497 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:45:02.043497 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:45:02.043497 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:45:02.043497 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:45:02.043497 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 01:45:02.043497 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 01:45:02.043497 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 01:45:02.043497 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 01:45:02.043497 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:45:02.043497 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:45:02.043497 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:45:02.043497 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:45:02.043497 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:45:02.043497 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:45:02.044356 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:45:02.044356 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:45:02.044356 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:45:02.044356 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:45:02.044356 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:45:02.044356 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:45:02.044356 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:45:02.044356 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:45:02.044356 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:45:02.044356 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:45:02.044356 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 01:45:02.044356 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:45:02.044356 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 01:45:02.044356 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:45:02.044356 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 530/1051] 10.222.0.26 () {56 vars in 1105 bytes} [Thu Jun 23 01:45:01 2022] GET /identity/v3/services/image => generated 86 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:45:02.050117 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5012f3-6a3f-4ef8-8df6-9254e304db4a None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:02.066392 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3c4505-567e-4b05-8693-0d78d7645105 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:02.104137 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62910d99-4db6-464b-bf84-7872322d0257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:02.104822 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62910d99-4db6-464b-bf84-7872322d0257 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:02.105103 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62910d99-4db6-464b-bf84-7872322d0257 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:02.105379 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62910d99-4db6-464b-bf84-7872322d0257 None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:02.106135 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62910d99-4db6-464b-bf84-7872322d0257 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:45:02.106444 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62910d99-4db6-464b-bf84-7872322d0257 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:02.107858 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62910d99-4db6-464b-bf84-7872322d0257 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:02.120971 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 522/1052] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:45:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:45:02.121789 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62910d99-4db6-464b-bf84-7872322d0257 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:02.159344 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5012f3-6a3f-4ef8-8df6-9254e304db4a None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:02.193507 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab6c0c3-91c1-414a-9f1b-fcbb3a4365a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:02.194346 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6c0c3-91c1-414a-9f1b-fcbb3a4365a5 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:02.194715 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6c0c3-91c1-414a-9f1b-fcbb3a4365a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:02.195086 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6c0c3-91c1-414a-9f1b-fcbb3a4365a5 None admin] PATH_INFO: `/v3/services` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:02.195833 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab6c0c3-91c1-414a-9f1b-fcbb3a4365a5 None admin] RBAC: Adding query filter params (name=image) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:45:02.196136 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab6c0c3-91c1-414a-9f1b-fcbb3a4365a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:02.197517 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab6c0c3-91c1-414a-9f1b-fcbb3a4365a5 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:02.206994 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 531/1053] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:45:02 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:02.208085 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab6c0c3-91c1-414a-9f1b-fcbb3a4365a5 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:02.215772 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:02.230206 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:02.233379 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62910d99-4db6-464b-bf84-7872322d0257 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:02.274525 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d132ee7-90de-4a93-81c2-b9f75e04cf50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:02.275514 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d132ee7-90de-4a93-81c2-b9f75e04cf50 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:02.275802 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d132ee7-90de-4a93-81c2-b9f75e04cf50 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:02.276079 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d132ee7-90de-4a93-81c2-b9f75e04cf50 None admin] PATH_INFO: `/v3/users/2cea8a88851c4db4bcfb5f702ec742f3` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:02.295501 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d132ee7-90de-4a93-81c2-b9f75e04cf50 None admin] RBAC: Authorizing `identity:get_user(user_id=2cea8a88851c4db4bcfb5f702ec742f3)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:02.296980 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d132ee7-90de-4a93-81c2-b9f75e04cf50 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:02.303420 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 523/1054] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:45:02 2022] GET /identity/v3/users/2cea8a88851c4db4bcfb5f702ec742f3 => generated 285 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:45:02.305067 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d132ee7-90de-4a93-81c2-b9f75e04cf50 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:02.326150 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab6c0c3-91c1-414a-9f1b-fcbb3a4365a5 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:02.362992 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b592f6-f546-418a-b13b-82db4ab00ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:02.363792 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b592f6-f546-418a-b13b-82db4ab00ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:02.364071 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b592f6-f546-418a-b13b-82db4ab00ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:02.364341 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b592f6-f546-418a-b13b-82db4ab00ef1 None admin] PATH_INFO: `/v3/services` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:02.365059 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b592f6-f546-418a-b13b-82db4ab00ef1 None admin] RBAC: Adding query filter params (type=image) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:45:02.365363 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b592f6-f546-418a-b13b-82db4ab00ef1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:02.367016 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b592f6-f546-418a-b13b-82db4ab00ef1 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:02.372687 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 532/1055] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:45:02 2022] GET /identity/v3/services?type=image => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:02.378051 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b592f6-f546-418a-b13b-82db4ab00ef1 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:02.410484 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d132ee7-90de-4a93-81c2-b9f75e04cf50 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:02.447473 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f276f16-655b-44cf-b218-9ce2f844736a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:02.448952 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f276f16-655b-44cf-b218-9ce2f844736a None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:02.449306 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f276f16-655b-44cf-b218-9ce2f844736a None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:02.449650 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f276f16-655b-44cf-b218-9ce2f844736a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:02.462678 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:02.470586 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:02.480309 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b592f6-f546-418a-b13b-82db4ab00ef1 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:02.517502 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f340150b-c583-4b7b-a97e-8186662062c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:02.518297 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f340150b-c583-4b7b-a97e-8186662062c0 None admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:02.518668 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f340150b-c583-4b7b-a97e-8186662062c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:02.519030 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f340150b-c583-4b7b-a97e-8186662062c0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:02.519884 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f340150b-c583-4b7b-a97e-8186662062c0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:02.521329 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f340150b-c583-4b7b-a97e-8186662062c0 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:02.526769 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f276f16-655b-44cf-b218-9ce2f844736a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:02.528348 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f276f16-655b-44cf-b218-9ce2f844736a None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:02.546077 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 533/1056] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Jun 23 01:45:02 2022] POST /identity/v3/endpoints => generated 327 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 01:45:02.651414 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 524/1057] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:45:02 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 23 01:45:02.657344 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f340150b-c583-4b7b-a97e-8186662062c0 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:02.774744 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f340150b-c583-4b7b-a97e-8186662062c0 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:02.808805 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c26717-52fe-4c22-ba4a-4091fc474712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:02.809663 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c26717-52fe-4c22-ba4a-4091fc474712 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:02.809942 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c26717-52fe-4c22-ba4a-4091fc474712 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:02.810190 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c26717-52fe-4c22-ba4a-4091fc474712 None admin] PATH_INFO: `/v3/projects/b2654f122e264abfa7a1a18204b2e2e4` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:02.811418 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c26717-52fe-4c22-ba4a-4091fc474712 None admin] RBAC: Authorizing `identity:get_project(project_id=b2654f122e264abfa7a1a18204b2e2e4)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:02.812855 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c26717-52fe-4c22-ba4a-4091fc474712 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:02.814813 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 534/1058] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:45:02 2022] GET /identity/v3/projects/b2654f122e264abfa7a1a18204b2e2e4 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:02.820915 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f276f16-655b-44cf-b218-9ce2f844736a None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:02.869876 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:02.927360 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f276f16-655b-44cf-b218-9ce2f844736a None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:02.964335 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dccbde-6444-42ae-99e3-32c0b0ebaf21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:02.964992 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dccbde-6444-42ae-99e3-32c0b0ebaf21 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:02.965270 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dccbde-6444-42ae-99e3-32c0b0ebaf21 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:02.965546 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dccbde-6444-42ae-99e3-32c0b0ebaf21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:02.966390 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dccbde-6444-42ae-99e3-32c0b0ebaf21 None admin] RBAC: Adding query filter params (role.id=7b20d65f49ee4a48894b6c060bb5d580, scope.project.id=b2654f122e264abfa7a1a18204b2e2e4, user.id=2cea8a88851c4db4bcfb5f702ec742f3) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:45:02.966703 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dccbde-6444-42ae-99e3-32c0b0ebaf21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:02.968169 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dccbde-6444-42ae-99e3-32c0b0ebaf21 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:02.974002 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 525/1059] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:45:02 2022] GET /identity/v3/role_assignments?user.id=2cea8a88851c4db4bcfb5f702ec742f3&scope.project.id=b2654f122e264abfa7a1a18204b2e2e4&role.id=7b20d65f49ee4a48894b6c060bb5d580 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:03.848722 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4816c22-ad01-4b43-9cea-2da4b12f39d4 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:03.849252 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4816c22-ad01-4b43-9cea-2da4b12f39d4 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:03.849252 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4816c22-ad01-4b43-9cea-2da4b12f39d4 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:03.850817 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 535/1060] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:45:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:45:03.856112 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d631e-01ae-480f-b90b-32c982098568 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:03.856523 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d631e-01ae-480f-b90b-32c982098568 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:03.856672 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d631e-01ae-480f-b90b-32c982098568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:03.873971 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-0b8d631e-01ae-480f-b90b-32c982098568 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:45:03.875157 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-0b8d631e-01ae-480f-b90b-32c982098568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:45:03.946579 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 526/1061] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:45:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:45:03.969135 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4816c22-ad01-4b43-9cea-2da4b12f39d4 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:03.974092 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:04.080338 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4816c22-ad01-4b43-9cea-2da4b12f39d4 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:04.114186 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c09829c-54b6-4270-b479-5e67794fb907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:04.114922 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c09829c-54b6-4270-b479-5e67794fb907 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:04.115204 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c09829c-54b6-4270-b479-5e67794fb907 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:04.115502 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c09829c-54b6-4270-b479-5e67794fb907 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:04.126286 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c09829c-54b6-4270-b479-5e67794fb907 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:45:04.134515 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-0c09829c-54b6-4270-b479-5e67794fb907 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 01:45:04.134515 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:45:04.134515 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:45:04.134515 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:45:04.134515 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:45:04.134515 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:45:04.134515 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:45:04.134515 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:45:04.134515 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:45:04.134515 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:45:04.134515 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:45:04.134515 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:45:04.134515 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:45:04.134515 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:45:04.134515 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:45:04.134515 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:45:04.135552 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:45:04.135552 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:45:04.135552 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:45:04.135552 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:45:04.135552 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:45:04.135552 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:45:04.135552 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:45:04.135552 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:45:04.135552 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:45:04.135552 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:45:04.135552 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:45:04.135552 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:45:04.135552 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:45:04.135552 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:45:04.135552 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:45:04.135552 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:45:04.136378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:45:04.136378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:45:04.136378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:45:04.136378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:45:04.136378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:45:04.136378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:45:04.136378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:45:04.136378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:45:04.136378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:45:04.136378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:45:04.136378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 01:45:04.136378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:45:04.136378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 536/1062] 10.222.0.26 () {56 vars in 1120 bytes} [Thu Jun 23 01:45:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:45:04.142166 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8d631e-01ae-480f-b90b-32c982098568 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:04.185326 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:04.248529 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8d631e-01ae-480f-b90b-32c982098568 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:04.286328 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2560a8-9419-4f6d-972b-2cb778b68e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:04.287483 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2560a8-9419-4f6d-972b-2cb778b68e74 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:04.287968 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2560a8-9419-4f6d-972b-2cb778b68e74 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:04.288442 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2560a8-9419-4f6d-972b-2cb778b68e74 None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:04.289389 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2560a8-9419-4f6d-972b-2cb778b68e74 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:45:04.289896 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2560a8-9419-4f6d-972b-2cb778b68e74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:04.291521 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2560a8-9419-4f6d-972b-2cb778b68e74 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:04.305233 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 527/1063] 10.222.0.26 () {56 vars in 1115 bytes} [Thu Jun 23 01:45:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:04.306778 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c09829c-54b6-4270-b479-5e67794fb907 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:04.370356 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:04.410527 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c09829c-54b6-4270-b479-5e67794fb907 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:04.445416 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264b066-8bb5-4e06-a613-384810533084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:04.446207 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264b066-8bb5-4e06-a613-384810533084 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:04.446481 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264b066-8bb5-4e06-a613-384810533084 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:04.446749 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264b066-8bb5-4e06-a613-384810533084 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:04.457505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e264b066-8bb5-4e06-a613-384810533084 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:04.459037 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e264b066-8bb5-4e06-a613-384810533084 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:04.476716 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-e264b066-8bb5-4e06-a613-384810533084 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:45:04.476716 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:45:04.476716 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:45:04.476716 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:45:04.476716 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:45:04.476716 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:45:04.476716 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:45:04.476716 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:45:04.476716 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:45:04.476716 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:45:04.476716 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:45:04.476716 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:45:04.476716 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:45:04.476716 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:45:04.476716 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:45:04.477680 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:45:04.477680 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:45:04.477680 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:45:04.477680 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:45:04.477680 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:45:04.477680 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:45:04.477680 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:45:04.477680 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:45:04.477680 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:45:04.477680 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:45:04.477680 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:45:04.477680 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:45:04.477680 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:45:04.477680 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:45:04.477680 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:45:04.477680 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:45:04.477680 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:45:04.478576 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:45:04.478576 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:45:04.478576 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:45:04.478576 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:45:04.482650 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 537/1064] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:45:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:45:04.484095 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2560a8-9419-4f6d-972b-2cb778b68e74 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:04.589464 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:04.593594 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2560a8-9419-4f6d-972b-2cb778b68e74 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:04.601040 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ab99a-aafb-4d15-bc9c-079af6e8e580 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:04.601329 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ab99a-aafb-4d15-bc9c-079af6e8e580 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:04.601604 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ab99a-aafb-4d15-bc9c-079af6e8e580 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:04.603133 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 538/1065] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:45:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 01:45:04.607936 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe74fdea-1e6e-4bbe-b65f-71d1310780e7 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:04.608223 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe74fdea-1e6e-4bbe-b65f-71d1310780e7 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:04.608511 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe74fdea-1e6e-4bbe-b65f-71d1310780e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:04.626158 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-fe74fdea-1e6e-4bbe-b65f-71d1310780e7 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:45:04.627359 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-fe74fdea-1e6e-4bbe-b65f-71d1310780e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:45:04.629662 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6841cb25-529f-42da-a2f0-3115ebb8f315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:04.630334 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841cb25-529f-42da-a2f0-3115ebb8f315 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:04.630610 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841cb25-529f-42da-a2f0-3115ebb8f315 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:04.630884 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841cb25-529f-42da-a2f0-3115ebb8f315 None admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:04.631633 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6841cb25-529f-42da-a2f0-3115ebb8f315 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:45:04.631937 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6841cb25-529f-42da-a2f0-3115ebb8f315 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:04.633299 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6841cb25-529f-42da-a2f0-3115ebb8f315 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:04.646307 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 528/1066] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:45:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:45:04.651959 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6841cb25-529f-42da-a2f0-3115ebb8f315 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:04.699284 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 539/1067] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:45:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 01:45:04.720082 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe74fdea-1e6e-4bbe-b65f-71d1310780e7 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:04.762197 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6841cb25-529f-42da-a2f0-3115ebb8f315 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:04.770886 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:04.799387 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897ceaa4-b240-4e21-9e6a-97d0e4d69a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:04.800180 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ceaa4-b240-4e21-9e6a-97d0e4d69a94 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:04.800823 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ceaa4-b240-4e21-9e6a-97d0e4d69a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:04.801108 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ceaa4-b240-4e21-9e6a-97d0e4d69a94 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:04.810771 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897ceaa4-b240-4e21-9e6a-97d0e4d69a94 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:04.812239 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897ceaa4-b240-4e21-9e6a-97d0e4d69a94 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:04.824818 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe74fdea-1e6e-4bbe-b65f-71d1310780e7 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:04.830602 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-897ceaa4-b240-4e21-9e6a-97d0e4d69a94 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 01:45:04.830602 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:45:04.830602 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:45:04.830602 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:45:04.830602 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:45:04.830602 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:45:04.830602 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:45:04.830602 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:45:04.830602 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:45:04.830602 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:45:04.830602 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:45:04.830602 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:45:04.830602 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 01:45:04.830602 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 01:45:04.830602 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 01:45:04.830602 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 01:45:04.831723 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:45:04.831723 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:45:04.831723 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 01:45:04.831723 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:45:04.831723 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 01:45:04.831723 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:45:04.831723 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:45:04.831723 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:45:04.831723 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:45:04.831723 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:45:04.831723 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:45:04.831723 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:45:04.831723 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:45:04.831723 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:45:04.831723 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:45:04.831723 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:45:04.831723 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:45:04.831723 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:45:04.837521 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:45:04.837521 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:45:04.837521 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 01:45:04.837521 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 01:45:04.837521 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 01:45:04.837521 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 01:45:04.837521 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 01:45:04.837521 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 01:45:04.837521 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 01:45:04.837521 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:45:04.837521 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 529/1068] 10.222.0.26 () {56 vars in 1093 bytes} [Thu Jun 23 01:45:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:45:04.837521 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897ceaa4-b240-4e21-9e6a-97d0e4d69a94 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:04.858666 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f287b2c-de93-413f-859b-05e39b7cf449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:04.859370 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f287b2c-de93-413f-859b-05e39b7cf449 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:04.859652 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f287b2c-de93-413f-859b-05e39b7cf449 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:04.859926 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f287b2c-de93-413f-859b-05e39b7cf449 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:04.865133 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:04.934501 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f287b2c-de93-413f-859b-05e39b7cf449 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:04.936307 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f287b2c-de93-413f-859b-05e39b7cf449 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:04.949692 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897ceaa4-b240-4e21-9e6a-97d0e4d69a94 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:04.983726 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:04.985923 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541e40c9-3225-4e49-bef9-abae3459c421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:04.986685 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e40c9-3225-4e49-bef9-abae3459c421 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:04.986964 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e40c9-3225-4e49-bef9-abae3459c421 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:04.987259 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e40c9-3225-4e49-bef9-abae3459c421 None admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:04.988105 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541e40c9-3225-4e49-bef9-abae3459c421 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:45:04.988410 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541e40c9-3225-4e49-bef9-abae3459c421 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:04.989837 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541e40c9-3225-4e49-bef9-abae3459c421 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:05.007137 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 530/1069] 10.222.0.26 () {56 vars in 1133 bytes} [Thu Jun 23 01:45:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:45:05.008609 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86085061-ed56-4ac5-8ea8-c2ec60f6c929 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:05.008890 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86085061-ed56-4ac5-8ea8-c2ec60f6c929 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:05.009168 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86085061-ed56-4ac5-8ea8-c2ec60f6c929 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:05.010389 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 531/1070] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:45:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:45:05.015100 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86085061-ed56-4ac5-8ea8-c2ec60f6c929 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:05.042514 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 540/1071] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:45:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 01:45:05.043849 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07496853-acdd-4742-b4d3-7f7e1c8de9c3 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:05.044142 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07496853-acdd-4742-b4d3-7f7e1c8de9c3 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:05.044417 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07496853-acdd-4742-b4d3-7f7e1c8de9c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:05.059965 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-07496853-acdd-4742-b4d3-7f7e1c8de9c3 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:45:05.061155 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-07496853-acdd-4742-b4d3-7f7e1c8de9c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:45:05.098790 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:05.121795 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86085061-ed56-4ac5-8ea8-c2ec60f6c929 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:05.126461 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 541/1072] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:45:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:45:05.127311 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07496853-acdd-4742-b4d3-7f7e1c8de9c3 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:05.159971 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b109c39-7045-4c1a-8e65-c48f4742d324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:05.160764 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b109c39-7045-4c1a-8e65-c48f4742d324 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:05.161050 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b109c39-7045-4c1a-8e65-c48f4742d324 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:05.161317 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b109c39-7045-4c1a-8e65-c48f4742d324 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:05.171963 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b109c39-7045-4c1a-8e65-c48f4742d324 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:05.173462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b109c39-7045-4c1a-8e65-c48f4742d324 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:05.191892 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-2b109c39-7045-4c1a-8e65-c48f4742d324 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:45:05.191892 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:45:05.191892 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:45:05.191892 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:45:05.191892 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:45:05.191892 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:45:05.191892 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:45:05.191892 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:45:05.191892 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:45:05.191892 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:45:05.191892 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:45:05.191892 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:45:05.191892 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:45:05.191892 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:45:05.191892 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:45:05.192764 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:45:05.192764 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:45:05.192764 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:45:05.192764 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:45:05.192764 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:45:05.192764 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:45:05.192764 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:45:05.192764 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:45:05.192764 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:45:05.192764 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:45:05.192764 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:45:05.192764 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:45:05.192764 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:45:05.192764 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:45:05.192764 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:45:05.192764 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:45:05.192764 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:45:05.193662 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:45:05.193662 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:45:05.193662 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:45:05.193662 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:45:05.195355 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 532/1073] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:45:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:45:05.196379 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b109c39-7045-4c1a-8e65-c48f4742d324 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:05.205308 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:05.228747 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07496853-acdd-4742-b4d3-7f7e1c8de9c3 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:05.261156 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea6344e-d0b2-4d8e-89c7-df3cb3f46a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:05.261954 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6344e-d0b2-4d8e-89c7-df3cb3f46a3d None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:05.262228 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6344e-d0b2-4d8e-89c7-df3cb3f46a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:05.262499 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6344e-d0b2-4d8e-89c7-df3cb3f46a3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:05.273643 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea6344e-d0b2-4d8e-89c7-df3cb3f46a3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:05.275111 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea6344e-d0b2-4d8e-89c7-df3cb3f46a3d None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:05.292083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-0ea6344e-d0b2-4d8e-89c7-df3cb3f46a3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:45:05.292083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:45:05.292083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:45:05.292083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:45:05.292083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:45:05.292083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:45:05.292083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:45:05.292083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:45:05.292083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:45:05.292083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:45:05.292083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:45:05.292083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:45:05.292083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:45:05.292083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:45:05.292083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:45:05.293091 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:45:05.293091 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:45:05.293091 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:45:05.293091 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:45:05.293091 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:45:05.293091 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:45:05.293091 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:45:05.293091 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:45:05.293091 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:45:05.293091 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:45:05.293091 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:45:05.293091 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:45:05.293091 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:45:05.293091 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:45:05.293091 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:45:05.293091 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:45:05.293091 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:45:05.294009 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:45:05.294009 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:45:05.294009 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:45:05.294009 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:45:05.294009 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 542/1074] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:45:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:45:05.294479 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea6344e-d0b2-4d8e-89c7-df3cb3f46a3d None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:05.298812 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b109c39-7045-4c1a-8e65-c48f4742d324 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:05.326921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:05.334124 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a99605e-6c00-44c1-95eb-5edcb6c3f09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:05.334918 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99605e-6c00-44c1-95eb-5edcb6c3f09e None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:05.335198 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99605e-6c00-44c1-95eb-5edcb6c3f09e None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:05.335496 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99605e-6c00-44c1-95eb-5edcb6c3f09e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:05.345544 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a99605e-6c00-44c1-95eb-5edcb6c3f09e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:45:05.353059 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-2a99605e-6c00-44c1-95eb-5edcb6c3f09e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:45:05.353059 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:45:05.353059 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:45:05.353059 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:45:05.353059 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:45:05.353059 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:45:05.353059 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:45:05.353059 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:45:05.353059 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:45:05.353059 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:45:05.353059 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:45:05.353059 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:45:05.353059 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:45:05.353059 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:45:05.353059 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:45:05.353059 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:45:05.354139 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:45:05.354139 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:45:05.354139 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:45:05.354139 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:45:05.354139 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:45:05.354139 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:45:05.354139 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:45:05.354139 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:45:05.354139 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:45:05.354139 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:45:05.354139 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:45:05.354139 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:45:05.354139 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:45:05.354139 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:45:05.354139 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:45:05.354139 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:45:05.360389 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:45:05.360389 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:45:05.360389 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:45:05.360389 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:45:05.360389 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:45:05.360389 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:45:05.360389 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:45:05.360389 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:45:05.360389 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:45:05.360389 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:45:05.360389 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:45:05.360389 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:45:05.360389 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 533/1075] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jun 23 01:45:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:45:05.360389 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a99605e-6c00-44c1-95eb-5edcb6c3f09e None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:05.393018 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea6344e-d0b2-4d8e-89c7-df3cb3f46a3d None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:05.418845 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:05.427494 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007f6bd1-fde4-49af-ad3a-0037492e11f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:05.428141 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f6bd1-fde4-49af-ad3a-0037492e11f3 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:05.428420 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f6bd1-fde4-49af-ad3a-0037492e11f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:05.428692 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f6bd1-fde4-49af-ad3a-0037492e11f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:05.429726 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007f6bd1-fde4-49af-ad3a-0037492e11f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:45:05.430024 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007f6bd1-fde4-49af-ad3a-0037492e11f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:05.431446 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007f6bd1-fde4-49af-ad3a-0037492e11f3 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:05.443101 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 543/1076] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:45:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:05.443834 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007f6bd1-fde4-49af-ad3a-0037492e11f3 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:05.465505 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a99605e-6c00-44c1-95eb-5edcb6c3f09e None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:05.498276 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767a791-e298-4ee6-83fb-5e53c3f57f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:05.498928 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767a791-e298-4ee6-83fb-5e53c3f57f89 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:05.499196 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767a791-e298-4ee6-83fb-5e53c3f57f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:05.499517 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767a791-e298-4ee6-83fb-5e53c3f57f89 None admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:05.500245 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7767a791-e298-4ee6-83fb-5e53c3f57f89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:45:05.500537 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7767a791-e298-4ee6-83fb-5e53c3f57f89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:05.501936 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7767a791-e298-4ee6-83fb-5e53c3f57f89 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:05.513875 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 534/1077] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:45:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:05.514670 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767a791-e298-4ee6-83fb-5e53c3f57f89 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:05.551256 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007f6bd1-fde4-49af-ad3a-0037492e11f3 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:05.574556 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:05.593061 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fa4fd4-2682-449c-8437-c5ee70298326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:05.594267 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa4fd4-2682-449c-8437-c5ee70298326 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:05.594267 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa4fd4-2682-449c-8437-c5ee70298326 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:05.594476 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa4fd4-2682-449c-8437-c5ee70298326 None admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:05.595263 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fa4fd4-2682-449c-8437-c5ee70298326 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:45:05.595577 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fa4fd4-2682-449c-8437-c5ee70298326 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:05.597006 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fa4fd4-2682-449c-8437-c5ee70298326 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:05.612451 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 544/1078] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:45:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:05.614073 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fa4fd4-2682-449c-8437-c5ee70298326 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:05.631579 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767a791-e298-4ee6-83fb-5e53c3f57f89 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:05.671594 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3477cf-e816-4c97-9e7f-f9386ee04e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:05.671801 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:05.671885 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3477cf-e816-4c97-9e7f-f9386ee04e93 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:05.672155 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3477cf-e816-4c97-9e7f-f9386ee04e93 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:05.672433 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3477cf-e816-4c97-9e7f-f9386ee04e93 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:05.682923 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3477cf-e816-4c97-9e7f-f9386ee04e93 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:05.684194 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3477cf-e816-4c97-9e7f-f9386ee04e93 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:05.699453 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-7e3477cf-e816-4c97-9e7f-f9386ee04e93 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:45:05.699453 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:45:05.699453 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:45:05.699453 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:45:05.699453 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:45:05.699453 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:45:05.699453 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:45:05.699453 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:45:05.699453 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:45:05.699453 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:45:05.699453 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:45:05.699453 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:45:05.699453 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 01:45:05.699453 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 01:45:05.699453 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 01:45:05.700340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 01:45:05.700340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:45:05.700340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:45:05.700340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:45:05.700340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:45:05.700340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:45:05.700340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:45:05.700340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:45:05.700340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:45:05.700340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:45:05.700340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:45:05.700340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:45:05.700340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:45:05.700340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:45:05.700340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:45:05.700340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:45:05.700340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:45:05.701213 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 01:45:05.701213 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 01:45:05.701213 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 01:45:05.701213 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 01:45:05.701213 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 01:45:05.701213 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 01:45:05.701213 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:45:05.701213 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:45:05.701213 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 535/1079] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:45:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:45:05.706603 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3477cf-e816-4c97-9e7f-f9386ee04e93 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:05.729351 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fa4fd4-2682-449c-8437-c5ee70298326 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:05.769319 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad90b08-a288-4a47-9d80-e51cd68d9a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:05.770199 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad90b08-a288-4a47-9d80-e51cd68d9a85 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:05.770472 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad90b08-a288-4a47-9d80-e51cd68d9a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:05.770744 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad90b08-a288-4a47-9d80-e51cd68d9a85 None admin] PATH_INFO: `/v3/users/2cea8a88851c4db4bcfb5f702ec742f3` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:05.772115 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad90b08-a288-4a47-9d80-e51cd68d9a85 None admin] RBAC: Authorizing `identity:get_user(user_id=2cea8a88851c4db4bcfb5f702ec742f3)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:05.773563 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad90b08-a288-4a47-9d80-e51cd68d9a85 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:05.775235 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 545/1080] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:45:05 2022] GET /identity/v3/users/2cea8a88851c4db4bcfb5f702ec742f3 => generated 285 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:05.781147 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad90b08-a288-4a47-9d80-e51cd68d9a85 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:05.824530 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3477cf-e816-4c97-9e7f-f9386ee04e93 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:05.834966 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:05.859071 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13388212-31c7-45de-87db-478544b22cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:05.859764 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13388212-31c7-45de-87db-478544b22cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:05.860037 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13388212-31c7-45de-87db-478544b22cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:05.860306 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13388212-31c7-45de-87db-478544b22cf3 None admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:05.861126 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13388212-31c7-45de-87db-478544b22cf3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:45:05.861429 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13388212-31c7-45de-87db-478544b22cf3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:05.862860 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13388212-31c7-45de-87db-478544b22cf3 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:05.874617 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 536/1081] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jun 23 01:45:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:45:05.880546 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13388212-31c7-45de-87db-478544b22cf3 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:05.881757 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad90b08-a288-4a47-9d80-e51cd68d9a85 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:05.899499 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:05.916448 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f88145-3743-480d-8652-f93038c3da5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:05.917298 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f88145-3743-480d-8652-f93038c3da5a None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:05.917580 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f88145-3743-480d-8652-f93038c3da5a None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:05.917853 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f88145-3743-480d-8652-f93038c3da5a None admin] PATH_INFO: `/v3/projects/b2654f122e264abfa7a1a18204b2e2e4` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:05.919044 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f88145-3743-480d-8652-f93038c3da5a None admin] RBAC: Authorizing `identity:get_project(project_id=b2654f122e264abfa7a1a18204b2e2e4)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:05.920514 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f88145-3743-480d-8652-f93038c3da5a None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:05.922416 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 546/1082] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:45:05 2022] GET /identity/v3/projects/b2654f122e264abfa7a1a18204b2e2e4 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:45:05.928070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f88145-3743-480d-8652-f93038c3da5a None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:05.985182 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13388212-31c7-45de-87db-478544b22cf3 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:06.021413 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bcbe7b-adb5-426e-bfac-cd1db5a05271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:06.022297 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bcbe7b-adb5-426e-bfac-cd1db5a05271 None admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:06.022572 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bcbe7b-adb5-426e-bfac-cd1db5a05271 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:06.022839 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bcbe7b-adb5-426e-bfac-cd1db5a05271 None admin] PATH_INFO: `/v3/projects/a44a5e0e3e2d4b8e9b9cec7b5b5c725c/users/ae4e83df33b64a3ebf28f54e2d94fe63/roles/f173bf8d88d34d82881f578a6c10a7ce` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:06.025229 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bcbe7b-adb5-426e-bfac-cd1db5a05271 None admin] RBAC: Authorizing `identity:create_grant(project_id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c, user_id=ae4e83df33b64a3ebf28f54e2d94fe63, role_id=f173bf8d88d34d82881f578a6c10a7ce)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:06.026686 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bcbe7b-adb5-426e-bfac-cd1db5a05271 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:06.033125 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f88145-3743-480d-8652-f93038c3da5a None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:06.041051 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 537/1083] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jun 23 01:45:05 2022] PUT /identity/v3/projects/a44a5e0e3e2d4b8e9b9cec7b5b5c725c/users/ae4e83df33b64a3ebf28f54e2d94fe63/roles/f173bf8d88d34d82881f578a6c10a7ce => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 01:45:06.073840 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed07b916-af36-45c6-bda0-39fa4ce2a17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:06.074749 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed07b916-af36-45c6-bda0-39fa4ce2a17f None admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:06.075025 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed07b916-af36-45c6-bda0-39fa4ce2a17f None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:06.075334 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed07b916-af36-45c6-bda0-39fa4ce2a17f None admin] PATH_INFO: `/v3/projects/b2654f122e264abfa7a1a18204b2e2e4/users/2cea8a88851c4db4bcfb5f702ec742f3/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:06.077526 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed07b916-af36-45c6-bda0-39fa4ce2a17f None admin] RBAC: Authorizing `identity:create_grant(project_id=b2654f122e264abfa7a1a18204b2e2e4, user_id=2cea8a88851c4db4bcfb5f702ec742f3, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:06.079001 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed07b916-af36-45c6-bda0-39fa4ce2a17f None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:06.096320 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 547/1084] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jun 23 01:45:05 2022] PUT /identity/v3/projects/b2654f122e264abfa7a1a18204b2e2e4/users/2cea8a88851c4db4bcfb5f702ec742f3/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 01:45:08.048607 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7968194-6b28-440e-838d-e43f5011fa93 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:08.049203 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7968194-6b28-440e-838d-e43f5011fa93 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:08.049203 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7968194-6b28-440e-838d-e43f5011fa93 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:08.050596 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 538/1085] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:45:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:45:08.056205 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf40da-4af6-4151-a94e-d745e4e08db6 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:08.056642 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf40da-4af6-4151-a94e-d745e4e08db6 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:08.056730 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf40da-4af6-4151-a94e-d745e4e08db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:08.073984 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-00bf40da-4af6-4151-a94e-d745e4e08db6 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:45:08.075166 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-00bf40da-4af6-4151-a94e-d745e4e08db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:45:08.088497 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf286d01-9360-4361-b25b-f3efafae708e None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:08.088876 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf286d01-9360-4361-b25b-f3efafae708e None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:08.089286 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf286d01-9360-4361-b25b-f3efafae708e None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:08.091362 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 539/1086] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:45:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:45:08.096214 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99ac83-5abf-43fe-9aea-649c2e71ee25 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:08.096493 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99ac83-5abf-43fe-9aea-649c2e71ee25 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:08.096767 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99ac83-5abf-43fe-9aea-649c2e71ee25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:08.113344 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-8e99ac83-5abf-43fe-9aea-649c2e71ee25 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:45:08.114554 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-8e99ac83-5abf-43fe-9aea-649c2e71ee25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:45:08.122134 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:08.140590 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:08.151260 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 548/1087] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:45:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:45:08.192342 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf40da-4af6-4151-a94e-d745e4e08db6 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:08.207640 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 540/1088] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:45:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 01:45:08.248243 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99ac83-5abf-43fe-9aea-649c2e71ee25 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:08.319574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf40da-4af6-4151-a94e-d745e4e08db6 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:08.354361 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd1c605-48b2-4c85-a160-9efbc7c2e153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:08.355181 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1c605-48b2-4c85-a160-9efbc7c2e153 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:08.355480 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1c605-48b2-4c85-a160-9efbc7c2e153 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:08.355761 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1c605-48b2-4c85-a160-9efbc7c2e153 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:08.358225 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99ac83-5abf-43fe-9aea-649c2e71ee25 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:08.365773 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd1c605-48b2-4c85-a160-9efbc7c2e153 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:45:08.373863 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-ecd1c605-48b2-4c85-a160-9efbc7c2e153 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 01:45:08.373863 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:45:08.373863 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:45:08.373863 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:45:08.373863 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:45:08.373863 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:45:08.373863 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:45:08.373863 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:45:08.373863 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:45:08.373863 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:45:08.373863 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:45:08.373863 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:45:08.373863 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:45:08.373863 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:45:08.373863 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:45:08.373863 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:45:08.374838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:45:08.374838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:45:08.374838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:45:08.374838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:45:08.374838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:45:08.374838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:45:08.374838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:45:08.374838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:45:08.374838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:45:08.374838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:45:08.374838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:45:08.374838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:45:08.374838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:45:08.374838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:45:08.374838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:45:08.374838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:45:08.376204 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:45:08.376204 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:45:08.376204 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:45:08.376204 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:45:08.376204 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:45:08.376204 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:45:08.376204 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:45:08.376204 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:45:08.376204 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:45:08.376204 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:45:08.376204 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 01:45:08.376204 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:45:08.376204 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 549/1089] 10.222.0.26 () {56 vars in 1120 bytes} [Thu Jun 23 01:45:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:45:08.380962 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd1c605-48b2-4c85-a160-9efbc7c2e153 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:08.393437 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0454730-6180-4f2b-afb0-9cc593d7d3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:08.394237 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0454730-6180-4f2b-afb0-9cc593d7d3d4 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:08.394516 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0454730-6180-4f2b-afb0-9cc593d7d3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:08.394787 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0454730-6180-4f2b-afb0-9cc593d7d3d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:08.404793 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0454730-6180-4f2b-afb0-9cc593d7d3d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:45:08.412376 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-e0454730-6180-4f2b-afb0-9cc593d7d3d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:45:08.412376 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:45:08.412376 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:45:08.412376 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:45:08.412376 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:45:08.412376 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:45:08.412376 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:45:08.412376 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:45:08.412376 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:45:08.412376 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:45:08.412376 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:45:08.412376 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:45:08.412376 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:45:08.412376 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:45:08.412376 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:45:08.412376 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:45:08.413427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:45:08.413427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:45:08.413427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:45:08.413427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:45:08.413427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:45:08.413427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:45:08.413427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:45:08.413427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:45:08.413427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:45:08.413427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:45:08.413427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:45:08.413427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:45:08.413427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:45:08.413427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:45:08.413427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:45:08.413427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:45:08.414282 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:45:08.414282 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:45:08.414282 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:45:08.414282 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:45:08.414282 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:45:08.414282 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:45:08.414282 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:45:08.414282 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:45:08.414282 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:45:08.414282 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:45:08.414282 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:45:08.414282 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:45:08.414282 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 541/1090] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jun 23 01:45:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:45:08.415931 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:08.419536 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0454730-6180-4f2b-afb0-9cc593d7d3d4 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:08.441683 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:08.478215 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd1c605-48b2-4c85-a160-9efbc7c2e153 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:08.510923 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078385cb-06e3-4c19-84b5-1eec8103e349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:08.511592 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078385cb-06e3-4c19-84b5-1eec8103e349 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:08.511868 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078385cb-06e3-4c19-84b5-1eec8103e349 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:08.512128 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078385cb-06e3-4c19-84b5-1eec8103e349 None admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:08.512863 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078385cb-06e3-4c19-84b5-1eec8103e349 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:45:08.513199 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078385cb-06e3-4c19-84b5-1eec8103e349 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:08.514564 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078385cb-06e3-4c19-84b5-1eec8103e349 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:08.518642 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0454730-6180-4f2b-afb0-9cc593d7d3d4 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:08.522953 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 550/1091] 10.222.0.26 () {56 vars in 1115 bytes} [Thu Jun 23 01:45:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:45:08.528642 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078385cb-06e3-4c19-84b5-1eec8103e349 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:08.552328 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5669ca3-f5a8-4a22-ae8d-77def247dfe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:08.553000 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5669ca3-f5a8-4a22-ae8d-77def247dfe0 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:08.553289 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5669ca3-f5a8-4a22-ae8d-77def247dfe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:08.553565 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5669ca3-f5a8-4a22-ae8d-77def247dfe0 None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:08.554641 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5669ca3-f5a8-4a22-ae8d-77def247dfe0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:45:08.554952 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5669ca3-f5a8-4a22-ae8d-77def247dfe0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:08.556359 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5669ca3-f5a8-4a22-ae8d-77def247dfe0 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:08.564454 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 542/1092] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:45:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:45:08.570120 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5669ca3-f5a8-4a22-ae8d-77def247dfe0 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:08.626827 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:08.630978 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078385cb-06e3-4c19-84b5-1eec8103e349 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:08.656644 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:08.664642 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bcd079-725b-4c45-9a76-8801caee1d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:08.665446 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bcd079-725b-4c45-9a76-8801caee1d7f None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:08.665730 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bcd079-725b-4c45-9a76-8801caee1d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:08.666007 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bcd079-725b-4c45-9a76-8801caee1d7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:08.674986 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5669ca3-f5a8-4a22-ae8d-77def247dfe0 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:08.676831 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bcd079-725b-4c45-9a76-8801caee1d7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:08.678303 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bcd079-725b-4c45-9a76-8801caee1d7f None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:08.695708 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-d8bcd079-725b-4c45-9a76-8801caee1d7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:45:08.695708 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:45:08.695708 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:45:08.695708 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:45:08.695708 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:45:08.695708 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:45:08.695708 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:45:08.695708 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:45:08.695708 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:45:08.695708 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:45:08.695708 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:45:08.695708 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:45:08.695708 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:45:08.695708 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:45:08.695708 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:45:08.696746 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:45:08.696746 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:45:08.696746 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:45:08.696746 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:45:08.696746 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:45:08.696746 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:45:08.696746 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:45:08.696746 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:45:08.696746 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:45:08.696746 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:45:08.696746 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:45:08.696746 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:45:08.696746 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:45:08.696746 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:45:08.696746 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:45:08.696746 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:45:08.696746 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:45:08.697775 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:45:08.697775 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:45:08.697775 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:45:08.697775 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:45:08.697775 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 551/1093] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:45:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:45:08.702627 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bcd079-725b-4c45-9a76-8801caee1d7f None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:08.707688 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979a606f-8eea-4ef2-a239-a36d8ff64add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:08.708557 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979a606f-8eea-4ef2-a239-a36d8ff64add None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:08.708834 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979a606f-8eea-4ef2-a239-a36d8ff64add None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:08.709102 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979a606f-8eea-4ef2-a239-a36d8ff64add None admin] PATH_INFO: `/v3/users/2cea8a88851c4db4bcfb5f702ec742f3` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:08.710334 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979a606f-8eea-4ef2-a239-a36d8ff64add None admin] RBAC: Authorizing `identity:get_user(user_id=2cea8a88851c4db4bcfb5f702ec742f3)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:08.711843 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979a606f-8eea-4ef2-a239-a36d8ff64add None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:08.715274 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 543/1094] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:45:08 2022] GET /identity/v3/users/2cea8a88851c4db4bcfb5f702ec742f3 => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:45:08.718901 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979a606f-8eea-4ef2-a239-a36d8ff64add None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:08.812556 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bcd079-725b-4c45-9a76-8801caee1d7f None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:08.824818 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979a606f-8eea-4ef2-a239-a36d8ff64add None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:08.845156 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb4c09-8d82-4c31-9b64-65ef14f83119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:08.845794 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb4c09-8d82-4c31-9b64-65ef14f83119 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:08.846076 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb4c09-8d82-4c31-9b64-65ef14f83119 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:08.846344 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb4c09-8d82-4c31-9b64-65ef14f83119 None admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:08.847129 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bb4c09-8d82-4c31-9b64-65ef14f83119 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:45:08.847465 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bb4c09-8d82-4c31-9b64-65ef14f83119 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:08.848844 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bb4c09-8d82-4c31-9b64-65ef14f83119 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:08.858567 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412cb80-b335-4e99-9d92-6e3afb6b33fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:08.859205 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412cb80-b335-4e99-9d92-6e3afb6b33fe None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:08.859504 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412cb80-b335-4e99-9d92-6e3afb6b33fe None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:08.859777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0412cb80-b335-4e99-9d92-6e3afb6b33fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:08.861039 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 552/1095] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:45:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:45:08.866570 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb4c09-8d82-4c31-9b64-65ef14f83119 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:08.878646 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:08.878841 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:08.928460 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0412cb80-b335-4e99-9d92-6e3afb6b33fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:08.930209 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0412cb80-b335-4e99-9d92-6e3afb6b33fe None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:08.967750 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb4c09-8d82-4c31-9b64-65ef14f83119 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:09.001684 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85a09d1-a0fb-4244-b6c1-276032ef195d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:09.002573 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a09d1-a0fb-4244-b6c1-276032ef195d None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:09.002915 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a09d1-a0fb-4244-b6c1-276032ef195d None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:09.003137 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a09d1-a0fb-4244-b6c1-276032ef195d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:09.011246 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85a09d1-a0fb-4244-b6c1-276032ef195d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:09.012732 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85a09d1-a0fb-4244-b6c1-276032ef195d None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:09.026609 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 544/1096] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:45:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 23 01:45:09.027246 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-f85a09d1-a0fb-4244-b6c1-276032ef195d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 01:45:09.027246 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:45:09.027246 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:45:09.027246 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:45:09.027246 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:45:09.027246 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:45:09.027246 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:45:09.027246 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:45:09.027246 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:45:09.027246 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:45:09.027246 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:45:09.027246 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:45:09.027246 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 01:45:09.027246 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 01:45:09.027246 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 01:45:09.027246 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 01:45:09.028378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:45:09.028378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:45:09.028378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 01:45:09.028378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:45:09.028378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 01:45:09.028378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:45:09.028378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:45:09.028378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:45:09.028378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:45:09.028378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:45:09.028378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:45:09.028378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:45:09.028378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:45:09.028378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:45:09.028378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:45:09.028378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:45:09.028378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:45:09.028378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:45:09.029343 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:45:09.029343 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:45:09.029343 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 01:45:09.029343 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 01:45:09.029343 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 01:45:09.029343 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 01:45:09.029343 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 01:45:09.029343 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 01:45:09.029343 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 01:45:09.029343 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:45:09.029343 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 553/1097] 10.222.0.26 () {56 vars in 1093 bytes} [Thu Jun 23 01:45:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:45:09.032484 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412cb80-b335-4e99-9d92-6e3afb6b33fe None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:09.034032 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85a09d1-a0fb-4244-b6c1-276032ef195d None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:09.082161 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:09.101064 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:09.131702 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85a09d1-a0fb-4244-b6c1-276032ef195d None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:09.132127 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0412cb80-b335-4e99-9d92-6e3afb6b33fe None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:09.166822 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa13d33-0644-4120-b13b-8d09ecae2a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:09.167049 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bff348-75b8-4077-9829-5bac14729a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:09.167689 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bff348-75b8-4077-9829-5bac14729a46 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:09.167787 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa13d33-0644-4120-b13b-8d09ecae2a38 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:09.167966 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa13d33-0644-4120-b13b-8d09ecae2a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:09.168075 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bff348-75b8-4077-9829-5bac14729a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:09.168239 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa13d33-0644-4120-b13b-8d09ecae2a38 None admin] PATH_INFO: `/v3/projects/b2654f122e264abfa7a1a18204b2e2e4` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:09.168329 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bff348-75b8-4077-9829-5bac14729a46 None admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:09.169096 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bff348-75b8-4077-9829-5bac14729a46 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:45:09.169403 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bff348-75b8-4077-9829-5bac14729a46 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:09.169576 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa13d33-0644-4120-b13b-8d09ecae2a38 None admin] RBAC: Authorizing `identity:get_project(project_id=b2654f122e264abfa7a1a18204b2e2e4)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:09.170834 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa13d33-0644-4120-b13b-8d09ecae2a38 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:09.170931 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bff348-75b8-4077-9829-5bac14729a46 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:09.172560 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 545/1098] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:45:09 2022] GET /identity/v3/projects/b2654f122e264abfa7a1a18204b2e2e4 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:09.178560 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa13d33-0644-4120-b13b-8d09ecae2a38 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:09.193445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 554/1099] 10.222.0.26 () {56 vars in 1133 bytes} [Thu Jun 23 01:45:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:09.194484 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bff348-75b8-4077-9829-5bac14729a46 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:09.279564 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa13d33-0644-4120-b13b-8d09ecae2a38 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:09.295101 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bff348-75b8-4077-9829-5bac14729a46 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:09.300576 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:09.315688 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc41f78-437d-4cdb-9255-8a805d68af07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:09.316345 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc41f78-437d-4cdb-9255-8a805d68af07 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:09.316626 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc41f78-437d-4cdb-9255-8a805d68af07 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:09.316902 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc41f78-437d-4cdb-9255-8a805d68af07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:09.317739 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc41f78-437d-4cdb-9255-8a805d68af07 None admin] RBAC: Adding query filter params (role.id=7b20d65f49ee4a48894b6c060bb5d580, scope.project.id=b2654f122e264abfa7a1a18204b2e2e4, user.id=2cea8a88851c4db4bcfb5f702ec742f3) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:45:09.318051 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc41f78-437d-4cdb-9255-8a805d68af07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:09.319514 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc41f78-437d-4cdb-9255-8a805d68af07 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:09.322018 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:09.325634 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 546/1100] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:45:09 2022] GET /identity/v3/role_assignments?user.id=2cea8a88851c4db4bcfb5f702ec742f3&scope.project.id=b2654f122e264abfa7a1a18204b2e2e4&role.id=7b20d65f49ee4a48894b6c060bb5d580 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:45:09.330050 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49968dd3-90ec-42db-8285-69e8538d664b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:09.330706 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49968dd3-90ec-42db-8285-69e8538d664b None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:09.330987 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49968dd3-90ec-42db-8285-69e8538d664b None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:09.331288 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49968dd3-90ec-42db-8285-69e8538d664b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:09.405329 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49968dd3-90ec-42db-8285-69e8538d664b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:09.406725 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49968dd3-90ec-42db-8285-69e8538d664b None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:09.514761 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 555/1101] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:45:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 01:45:09.520610 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc41f78-437d-4cdb-9255-8a805d68af07 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:09.624262 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc41f78-437d-4cdb-9255-8a805d68af07 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:09.658597 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f169c-b75f-4dea-8f13-606b9e85c30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:09.659333 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f169c-b75f-4dea-8f13-606b9e85c30b None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:09.659610 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f169c-b75f-4dea-8f13-606b9e85c30b None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:09.659877 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f169c-b75f-4dea-8f13-606b9e85c30b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:09.670842 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96f169c-b75f-4dea-8f13-606b9e85c30b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:09.672309 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96f169c-b75f-4dea-8f13-606b9e85c30b None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:09.689917 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-a96f169c-b75f-4dea-8f13-606b9e85c30b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:45:09.689917 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:45:09.689917 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:45:09.689917 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:45:09.689917 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:45:09.689917 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:45:09.689917 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:45:09.689917 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:45:09.689917 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:45:09.689917 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:45:09.689917 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:45:09.689917 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:45:09.689917 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:45:09.689917 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:45:09.689917 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:45:09.690827 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:45:09.690827 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:45:09.690827 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:45:09.690827 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:45:09.690827 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:45:09.690827 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:45:09.690827 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:45:09.690827 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:45:09.690827 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:45:09.690827 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:45:09.690827 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:45:09.690827 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:45:09.690827 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:45:09.690827 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:45:09.690827 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:45:09.690827 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:45:09.690827 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:45:09.691752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:45:09.691752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:45:09.691752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:45:09.691752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:45:09.691752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 547/1102] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:45:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:45:09.696930 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49968dd3-90ec-42db-8285-69e8538d664b None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:09.724923 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:09.798177 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49968dd3-90ec-42db-8285-69e8538d664b None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:09.836024 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee2d24d-4846-46cb-bbfc-d46643e58076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:09.836679 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee2d24d-4846-46cb-bbfc-d46643e58076 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:09.836964 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee2d24d-4846-46cb-bbfc-d46643e58076 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:09.837215 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee2d24d-4846-46cb-bbfc-d46643e58076 None admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:09.837933 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee2d24d-4846-46cb-bbfc-d46643e58076 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:45:09.838233 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee2d24d-4846-46cb-bbfc-d46643e58076 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:09.839642 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee2d24d-4846-46cb-bbfc-d46643e58076 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:09.851127 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 556/1103] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:45:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:09.856864 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f169c-b75f-4dea-8f13-606b9e85c30b None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:09.892421 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:09.959984 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f169c-b75f-4dea-8f13-606b9e85c30b None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:09.995362 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8dc542-90fb-43f4-96c6-348e1383ae0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:09.996161 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8dc542-90fb-43f4-96c6-348e1383ae0d None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:09.996440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8dc542-90fb-43f4-96c6-348e1383ae0d None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:09.996714 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8dc542-90fb-43f4-96c6-348e1383ae0d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:10.003963 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8dc542-90fb-43f4-96c6-348e1383ae0d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:10.005408 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8dc542-90fb-43f4-96c6-348e1383ae0d None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:10.019106 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-4c8dc542-90fb-43f4-96c6-348e1383ae0d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:45:10.019106 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:45:10.019106 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:45:10.019106 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:45:10.019106 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:45:10.019106 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:45:10.019106 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:45:10.019106 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:45:10.019106 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:45:10.019106 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:45:10.019106 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:45:10.019106 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:45:10.019106 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 01:45:10.019106 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 01:45:10.019106 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 01:45:10.020020 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 01:45:10.020020 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:45:10.020020 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:45:10.020020 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:45:10.020020 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:45:10.020020 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:45:10.020020 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:45:10.020020 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:45:10.020020 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:45:10.020020 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:45:10.020020 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:45:10.020020 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:45:10.020020 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:45:10.020020 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:45:10.020020 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:45:10.020020 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:45:10.020020 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:45:10.020880 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 01:45:10.020880 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 01:45:10.020880 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 01:45:10.020880 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 01:45:10.020880 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 01:45:10.020880 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 01:45:10.020880 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:45:10.020880 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:45:10.020880 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 548/1104] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:45:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:45:10.026323 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee2d24d-4846-46cb-bbfc-d46643e58076 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:10.118242 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:10.131830 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee2d24d-4846-46cb-bbfc-d46643e58076 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:10.166134 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a676ffe1-5dc1-4d2b-a127-7f510991b09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:10.166791 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676ffe1-5dc1-4d2b-a127-7f510991b09a None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:10.167071 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676ffe1-5dc1-4d2b-a127-7f510991b09a None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:10.167373 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676ffe1-5dc1-4d2b-a127-7f510991b09a None admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:10.168171 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a676ffe1-5dc1-4d2b-a127-7f510991b09a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:45:10.168477 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a676ffe1-5dc1-4d2b-a127-7f510991b09a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:10.169921 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a676ffe1-5dc1-4d2b-a127-7f510991b09a None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:10.182523 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 557/1105] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jun 23 01:45:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:10.188825 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8dc542-90fb-43f4-96c6-348e1383ae0d None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:10.287815 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:10.295437 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8dc542-90fb-43f4-96c6-348e1383ae0d None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:10.328783 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be81da59-8a5f-4948-a106-4847e2f48510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:10.329440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be81da59-8a5f-4948-a106-4847e2f48510 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:10.329717 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be81da59-8a5f-4948-a106-4847e2f48510 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:10.329991 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be81da59-8a5f-4948-a106-4847e2f48510 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:10.330831 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be81da59-8a5f-4948-a106-4847e2f48510 None admin] RBAC: Adding query filter params (role.id=f173bf8d88d34d82881f578a6c10a7ce, scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c, user.id=ae4e83df33b64a3ebf28f54e2d94fe63) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:45:10.331145 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be81da59-8a5f-4948-a106-4847e2f48510 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:10.332608 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be81da59-8a5f-4948-a106-4847e2f48510 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:10.338307 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 549/1106] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:45:10 2022] GET /identity/v3/role_assignments?user.id=ae4e83df33b64a3ebf28f54e2d94fe63&scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c&role.id=f173bf8d88d34d82881f578a6c10a7ce => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:11.366854 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3b61e-9071-48c0-8afe-5f4c82362903 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:11.367681 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3b61e-9071-48c0-8afe-5f4c82362903 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:11.367820 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af3b61e-9071-48c0-8afe-5f4c82362903 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:11.369986 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 558/1107] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:45:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:45:11.378464 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efdc7d-e6de-45d6-90c4-04b2a9ee5b03 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:11.378881 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efdc7d-e6de-45d6-90c4-04b2a9ee5b03 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:11.379045 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efdc7d-e6de-45d6-90c4-04b2a9ee5b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:11.401772 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-b8efdc7d-e6de-45d6-90c4-04b2a9ee5b03 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:45:11.402954 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-b8efdc7d-e6de-45d6-90c4-04b2a9ee5b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:45:11.472599 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 550/1108] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:45:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:45:11.502480 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af3b61e-9071-48c0-8afe-5f4c82362903 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:11.614793 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af3b61e-9071-48c0-8afe-5f4c82362903 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:11.649710 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb936e8-00b0-4d0a-a20a-e5a2b4b45dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:11.650488 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb936e8-00b0-4d0a-a20a-e5a2b4b45dae None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:11.650766 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb936e8-00b0-4d0a-a20a-e5a2b4b45dae None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:11.651042 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb936e8-00b0-4d0a-a20a-e5a2b4b45dae None admin] PATH_INFO: `/v3/domains/default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:11.652305 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb936e8-00b0-4d0a-a20a-e5a2b4b45dae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:11.653748 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb936e8-00b0-4d0a-a20a-e5a2b4b45dae None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:11.655415 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 559/1109] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:45:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:11.661636 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8efdc7d-e6de-45d6-90c4-04b2a9ee5b03 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:11.755496 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:11.776380 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8efdc7d-e6de-45d6-90c4-04b2a9ee5b03 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:11.813463 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf2e153-87a9-4a29-9791-e3b6094bae6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:11.814165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2e153-87a9-4a29-9791-e3b6094bae6c None admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:11.814486 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2e153-87a9-4a29-9791-e3b6094bae6c None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:11.814764 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2e153-87a9-4a29-9791-e3b6094bae6c None admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:11.815502 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf2e153-87a9-4a29-9791-e3b6094bae6c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:11.816949 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf2e153-87a9-4a29-9791-e3b6094bae6c None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:11.841921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 551/1110] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Jun 23 01:45:11 2022] POST /identity/v3/users => generated 286 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:45:13.881890 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c24bf1-a257-4d31-9f4f-37e8a1c7e1c3 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:13.882451 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c24bf1-a257-4d31-9f4f-37e8a1c7e1c3 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:13.882451 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c24bf1-a257-4d31-9f4f-37e8a1c7e1c3 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:13.883869 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 560/1111] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:45:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:45:13.890437 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ae5c8-013e-45a9-9c23-60d4f86b510d None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:13.891118 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ae5c8-013e-45a9-9c23-60d4f86b510d None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:13.891347 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ae5c8-013e-45a9-9c23-60d4f86b510d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:13.914772 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-474ae5c8-013e-45a9-9c23-60d4f86b510d None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:45:13.915993 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-474ae5c8-013e-45a9-9c23-60d4f86b510d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:45:13.985704 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 552/1112] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:45:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:45:14.011622 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c24bf1-a257-4d31-9f4f-37e8a1c7e1c3 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:14.037833 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:14.135450 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c24bf1-a257-4d31-9f4f-37e8a1c7e1c3 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:14.170543 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667b2c59-d35d-47fa-80eb-dcd8756b8355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:14.171416 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b2c59-d35d-47fa-80eb-dcd8756b8355 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:14.171702 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b2c59-d35d-47fa-80eb-dcd8756b8355 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:14.171980 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b2c59-d35d-47fa-80eb-dcd8756b8355 None admin] PATH_INFO: `/v3/roles/1c7adeca27524e258f7ddd2c107e43e1` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:14.173627 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667b2c59-d35d-47fa-80eb-dcd8756b8355 None admin] RBAC: Authorizing `identity:get_role(role_id=1c7adeca27524e258f7ddd2c107e43e1)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:14.175010 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667b2c59-d35d-47fa-80eb-dcd8756b8355 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:14.176421 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 561/1113] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:45:14 2022] GET /identity/v3/roles/1c7adeca27524e258f7ddd2c107e43e1 => generated 223 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:14.185830 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ae5c8-013e-45a9-9c23-60d4f86b510d None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:14.266138 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:14.293414 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ae5c8-013e-45a9-9c23-60d4f86b510d None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:14.328500 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a49782-9880-4023-a87d-9037f2b49fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:14.329447 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a49782-9880-4023-a87d-9037f2b49fa5 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:14.329725 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a49782-9880-4023-a87d-9037f2b49fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:14.329990 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a49782-9880-4023-a87d-9037f2b49fa5 None admin] PATH_INFO: `/v3/users/2b0a697585a54c45a2450ee26f21ffc7` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:14.343849 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a49782-9880-4023-a87d-9037f2b49fa5 None admin] RBAC: Authorizing `identity:get_user(user_id=2b0a697585a54c45a2450ee26f21ffc7)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:14.345315 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a49782-9880-4023-a87d-9037f2b49fa5 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:14.346962 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 553/1114] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:45:14 2022] GET /identity/v3/users/2b0a697585a54c45a2450ee26f21ffc7 => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:14.353428 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667b2c59-d35d-47fa-80eb-dcd8756b8355 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:14.451507 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:14.471130 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667b2c59-d35d-47fa-80eb-dcd8756b8355 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:14.507568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ef3ec-f516-47c5-9050-4b7663f11e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:14.508202 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ef3ec-f516-47c5-9050-4b7663f11e30 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:14.508480 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ef3ec-f516-47c5-9050-4b7663f11e30 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:14.508754 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ef3ec-f516-47c5-9050-4b7663f11e30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:14.587190 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4ef3ec-f516-47c5-9050-4b7663f11e30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:14.588666 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4ef3ec-f516-47c5-9050-4b7663f11e30 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:14.670498 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:14.688200 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 562/1115] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:45:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 01:45:14.697696 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a49782-9880-4023-a87d-9037f2b49fa5 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:14.809428 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a49782-9880-4023-a87d-9037f2b49fa5 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:14.841747 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:14.847746 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e75c419-feb6-4348-8815-8dfa5ed9c4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:14.848615 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e75c419-feb6-4348-8815-8dfa5ed9c4da None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:14.848895 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e75c419-feb6-4348-8815-8dfa5ed9c4da None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:14.849172 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e75c419-feb6-4348-8815-8dfa5ed9c4da None admin] PATH_INFO: `/v3/projects/b2654f122e264abfa7a1a18204b2e2e4` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:14.850387 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e75c419-feb6-4348-8815-8dfa5ed9c4da None admin] RBAC: Authorizing `identity:get_project(project_id=b2654f122e264abfa7a1a18204b2e2e4)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:14.851889 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e75c419-feb6-4348-8815-8dfa5ed9c4da None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:14.853586 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 554/1116] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:45:14 2022] GET /identity/v3/projects/b2654f122e264abfa7a1a18204b2e2e4 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:14.859870 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ef3ec-f516-47c5-9050-4b7663f11e30 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:14.971778 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ef3ec-f516-47c5-9050-4b7663f11e30 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:15.007228 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d89000-6f83-41b3-8c6a-c5f2b9df4d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:15.007955 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d89000-6f83-41b3-8c6a-c5f2b9df4d87 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:15.008234 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d89000-6f83-41b3-8c6a-c5f2b9df4d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:15.008508 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d89000-6f83-41b3-8c6a-c5f2b9df4d87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:15.009344 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d89000-6f83-41b3-8c6a-c5f2b9df4d87 None admin] RBAC: Adding query filter params (role.id=1c7adeca27524e258f7ddd2c107e43e1, scope.project.id=b2654f122e264abfa7a1a18204b2e2e4, user.id=2b0a697585a54c45a2450ee26f21ffc7) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:45:15.009657 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d89000-6f83-41b3-8c6a-c5f2b9df4d87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:15.011097 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d89000-6f83-41b3-8c6a-c5f2b9df4d87 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:15.016770 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 563/1117] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:45:14 2022] GET /identity/v3/role_assignments?user.id=2b0a697585a54c45a2450ee26f21ffc7&scope.project.id=b2654f122e264abfa7a1a18204b2e2e4&role.id=1c7adeca27524e258f7ddd2c107e43e1 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:17.005714 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5125b568-2dd3-419a-80ff-ea577b96fa68 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:17.006295 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5125b568-2dd3-419a-80ff-ea577b96fa68 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:17.006295 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5125b568-2dd3-419a-80ff-ea577b96fa68 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:17.007689 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 555/1118] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:45:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:45:17.013947 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79e226-1d1f-4e7a-a36b-c22b60541932 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:17.014647 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79e226-1d1f-4e7a-a36b-c22b60541932 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:17.014797 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79e226-1d1f-4e7a-a36b-c22b60541932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:17.039447 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-ca79e226-1d1f-4e7a-a36b-c22b60541932 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:45:17.040628 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-ca79e226-1d1f-4e7a-a36b-c22b60541932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:45:17.085918 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:17.113549 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 564/1119] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:45:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:45:17.137835 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5125b568-2dd3-419a-80ff-ea577b96fa68 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:17.244602 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5125b568-2dd3-419a-80ff-ea577b96fa68 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:17.282096 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ba541-e481-4aa5-97fd-1067508d9097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:17.282888 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ba541-e481-4aa5-97fd-1067508d9097 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:17.283162 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ba541-e481-4aa5-97fd-1067508d9097 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:17.283468 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ba541-e481-4aa5-97fd-1067508d9097 None admin] PATH_INFO: `/v3/roles/1c7adeca27524e258f7ddd2c107e43e1` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:17.285001 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902ba541-e481-4aa5-97fd-1067508d9097 None admin] RBAC: Authorizing `identity:get_role(role_id=1c7adeca27524e258f7ddd2c107e43e1)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:17.286378 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902ba541-e481-4aa5-97fd-1067508d9097 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:17.287796 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 556/1120] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:45:17 2022] GET /identity/v3/roles/1c7adeca27524e258f7ddd2c107e43e1 => generated 223 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:17.293900 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca79e226-1d1f-4e7a-a36b-c22b60541932 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:17.396073 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca79e226-1d1f-4e7a-a36b-c22b60541932 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:17.431895 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9412c21-f1dc-4582-b0ad-6752b5c977c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:17.432751 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9412c21-f1dc-4582-b0ad-6752b5c977c9 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:17.433026 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9412c21-f1dc-4582-b0ad-6752b5c977c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:17.433293 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9412c21-f1dc-4582-b0ad-6752b5c977c9 None admin] PATH_INFO: `/v3/users/2b0a697585a54c45a2450ee26f21ffc7` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:17.434521 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9412c21-f1dc-4582-b0ad-6752b5c977c9 None admin] RBAC: Authorizing `identity:get_user(user_id=2b0a697585a54c45a2450ee26f21ffc7)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:17.435986 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9412c21-f1dc-4582-b0ad-6752b5c977c9 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:17.437589 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 565/1121] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:45:17 2022] GET /identity/v3/users/2b0a697585a54c45a2450ee26f21ffc7 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:17.446681 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ba541-e481-4aa5-97fd-1067508d9097 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:17.519568 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:17.554167 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ba541-e481-4aa5-97fd-1067508d9097 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:17.596280 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4d3def-b20a-470f-82e6-4faebf3b6990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:17.597125 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d3def-b20a-470f-82e6-4faebf3b6990 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:17.597400 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d3def-b20a-470f-82e6-4faebf3b6990 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:17.597669 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d3def-b20a-470f-82e6-4faebf3b6990 None admin] PATH_INFO: `/v3/projects/b2654f122e264abfa7a1a18204b2e2e4` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:17.598905 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4d3def-b20a-470f-82e6-4faebf3b6990 None admin] RBAC: Authorizing `identity:get_project(project_id=b2654f122e264abfa7a1a18204b2e2e4)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:17.600381 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4d3def-b20a-470f-82e6-4faebf3b6990 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:17.602042 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 557/1122] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:45:17 2022] GET /identity/v3/projects/b2654f122e264abfa7a1a18204b2e2e4 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:17.608081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9412c21-f1dc-4582-b0ad-6752b5c977c9 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:17.657049 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:17.724315 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9412c21-f1dc-4582-b0ad-6752b5c977c9 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:17.760534 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e394ad-db01-4ffe-bd8d-856158a33660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:17.761404 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e394ad-db01-4ffe-bd8d-856158a33660 None admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:17.761687 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e394ad-db01-4ffe-bd8d-856158a33660 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:17.761961 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e394ad-db01-4ffe-bd8d-856158a33660 None admin] PATH_INFO: `/v3/projects/b2654f122e264abfa7a1a18204b2e2e4/users/2b0a697585a54c45a2450ee26f21ffc7/roles/1c7adeca27524e258f7ddd2c107e43e1` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:17.764253 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e394ad-db01-4ffe-bd8d-856158a33660 None admin] RBAC: Authorizing `identity:create_grant(project_id=b2654f122e264abfa7a1a18204b2e2e4, user_id=2b0a697585a54c45a2450ee26f21ffc7, role_id=1c7adeca27524e258f7ddd2c107e43e1)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:17.765699 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e394ad-db01-4ffe-bd8d-856158a33660 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:17.784766 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 566/1123] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jun 23 01:45:17 2022] PUT /identity/v3/projects/b2654f122e264abfa7a1a18204b2e2e4/users/2b0a697585a54c45a2450ee26f21ffc7/roles/1c7adeca27524e258f7ddd2c107e43e1 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:45:19.732110 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781938a-1df9-4916-8918-4071fe360e1d None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:19.732666 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781938a-1df9-4916-8918-4071fe360e1d None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:19.732666 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781938a-1df9-4916-8918-4071fe360e1d None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:19.734761 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 558/1124] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:45:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:45:19.740608 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f411d-c081-4f87-8f82-adae1eca8485 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:19.741066 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f411d-c081-4f87-8f82-adae1eca8485 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:19.741164 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f411d-c081-4f87-8f82-adae1eca8485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:19.759462 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-fa6f411d-c081-4f87-8f82-adae1eca8485 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:45:19.760638 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-fa6f411d-c081-4f87-8f82-adae1eca8485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:45:19.828214 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 567/1125] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:45:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 01:45:19.852059 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5781938a-1df9-4916-8918-4071fe360e1d None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:19.978417 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5781938a-1df9-4916-8918-4071fe360e1d None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:20.022873 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821b5c4a-8e21-4c22-8b17-1ff3133a91ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:20.023683 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b5c4a-8e21-4c22-8b17-1ff3133a91ac None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:20.023958 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b5c4a-8e21-4c22-8b17-1ff3133a91ac None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:20.024239 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b5c4a-8e21-4c22-8b17-1ff3133a91ac None admin] PATH_INFO: `/v3/roles/1c7adeca27524e258f7ddd2c107e43e1` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:20.026174 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821b5c4a-8e21-4c22-8b17-1ff3133a91ac None admin] RBAC: Authorizing `identity:get_role(role_id=1c7adeca27524e258f7ddd2c107e43e1)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:20.027583 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821b5c4a-8e21-4c22-8b17-1ff3133a91ac None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:20.028955 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 559/1126] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:45:19 2022] GET /identity/v3/roles/1c7adeca27524e258f7ddd2c107e43e1 => generated 223 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:20.036179 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f411d-c081-4f87-8f82-adae1eca8485 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:20.082588 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:20.156964 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f411d-c081-4f87-8f82-adae1eca8485 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:20.194959 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7399bd74-9ce6-48df-b4a6-81ceb804fb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:20.195850 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399bd74-9ce6-48df-b4a6-81ceb804fb9a None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:20.196135 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399bd74-9ce6-48df-b4a6-81ceb804fb9a None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:20.196406 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399bd74-9ce6-48df-b4a6-81ceb804fb9a None admin] PATH_INFO: `/v3/users/2b0a697585a54c45a2450ee26f21ffc7` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:20.197723 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7399bd74-9ce6-48df-b4a6-81ceb804fb9a None admin] RBAC: Authorizing `identity:get_user(user_id=2b0a697585a54c45a2450ee26f21ffc7)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:20.199180 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7399bd74-9ce6-48df-b4a6-81ceb804fb9a None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:20.200842 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 568/1127] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:45:20 2022] GET /identity/v3/users/2b0a697585a54c45a2450ee26f21ffc7 => generated 286 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:20.207674 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821b5c4a-8e21-4c22-8b17-1ff3133a91ac None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:20.213936 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:20.316709 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821b5c4a-8e21-4c22-8b17-1ff3133a91ac None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:20.353319 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a67db-d0d9-4085-a909-5632ef12bc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:20.353970 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a67db-d0d9-4085-a909-5632ef12bc06 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:20.354250 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a67db-d0d9-4085-a909-5632ef12bc06 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:20.354527 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a67db-d0d9-4085-a909-5632ef12bc06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:20.423739 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41a67db-d0d9-4085-a909-5632ef12bc06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:20.425198 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41a67db-d0d9-4085-a909-5632ef12bc06 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:20.427894 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:20.537529 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 560/1128] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:45:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 01:45:20.544058 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7399bd74-9ce6-48df-b4a6-81ceb804fb9a None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:20.645683 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7399bd74-9ce6-48df-b4a6-81ceb804fb9a None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:20.648009 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:20.681510 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade5992-ad39-42c2-9c97-fcc4b075ec83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:20.682348 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade5992-ad39-42c2-9c97-fcc4b075ec83 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:20.682617 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade5992-ad39-42c2-9c97-fcc4b075ec83 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:20.682884 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade5992-ad39-42c2-9c97-fcc4b075ec83 None admin] PATH_INFO: `/v3/projects/b2654f122e264abfa7a1a18204b2e2e4` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:20.684141 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade5992-ad39-42c2-9c97-fcc4b075ec83 None admin] RBAC: Authorizing `identity:get_project(project_id=b2654f122e264abfa7a1a18204b2e2e4)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:20.685573 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade5992-ad39-42c2-9c97-fcc4b075ec83 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:20.687240 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 569/1129] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:45:20 2022] GET /identity/v3/projects/b2654f122e264abfa7a1a18204b2e2e4 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:20.693575 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a67db-d0d9-4085-a909-5632ef12bc06 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:20.791913 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a67db-d0d9-4085-a909-5632ef12bc06 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:20.794293 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:20.828170 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1328d50d-22b4-4884-b7b8-372d0748a8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:20.828822 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1328d50d-22b4-4884-b7b8-372d0748a8d9 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:20.829099 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1328d50d-22b4-4884-b7b8-372d0748a8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:20.829400 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1328d50d-22b4-4884-b7b8-372d0748a8d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:20.830233 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1328d50d-22b4-4884-b7b8-372d0748a8d9 None admin] RBAC: Adding query filter params (role.id=1c7adeca27524e258f7ddd2c107e43e1, scope.project.id=b2654f122e264abfa7a1a18204b2e2e4, user.id=2b0a697585a54c45a2450ee26f21ffc7) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:45:20.830588 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1328d50d-22b4-4884-b7b8-372d0748a8d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:20.832700 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1328d50d-22b4-4884-b7b8-372d0748a8d9 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:20.840717 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 561/1130] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:45:20 2022] GET /identity/v3/role_assignments?user.id=2b0a697585a54c45a2450ee26f21ffc7&scope.project.id=b2654f122e264abfa7a1a18204b2e2e4&role.id=1c7adeca27524e258f7ddd2c107e43e1 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:22.879005 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c9cb82-06bf-44dc-981b-91633da241af None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:22.879615 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c9cb82-06bf-44dc-981b-91633da241af None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:22.879615 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c9cb82-06bf-44dc-981b-91633da241af None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:22.880973 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 570/1131] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:45:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:45:22.887272 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8f522-42d2-4b50-84d9-52d51b8cf504 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:22.887999 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8f522-42d2-4b50-84d9-52d51b8cf504 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:22.888144 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8f522-42d2-4b50-84d9-52d51b8cf504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:22.912546 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-3ed8f522-42d2-4b50-84d9-52d51b8cf504 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:45:22.913728 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-3ed8f522-42d2-4b50-84d9-52d51b8cf504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:45:22.988342 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 562/1132] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:45:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:45:23.012068 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c9cb82-06bf-44dc-981b-91633da241af None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:23.127205 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c9cb82-06bf-44dc-981b-91633da241af None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:23.162300 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93641d38-8843-472f-b699-173833f711fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:23.162999 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93641d38-8843-472f-b699-173833f711fc None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:23.163321 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93641d38-8843-472f-b699-173833f711fc None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:23.163573 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93641d38-8843-472f-b699-173833f711fc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:23.164859 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93641d38-8843-472f-b699-173833f711fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:23.166297 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93641d38-8843-472f-b699-173833f711fc None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:23.167924 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 571/1133] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:45:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:23.177127 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed8f522-42d2-4b50-84d9-52d51b8cf504 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:23.283270 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:23.289370 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed8f522-42d2-4b50-84d9-52d51b8cf504 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:23.328835 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb15a4fc-60ea-4269-8bef-c42fd0372078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:23.329488 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15a4fc-60ea-4269-8bef-c42fd0372078 None admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:23.329765 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15a4fc-60ea-4269-8bef-c42fd0372078 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:23.330032 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15a4fc-60ea-4269-8bef-c42fd0372078 None admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:23.330755 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb15a4fc-60ea-4269-8bef-c42fd0372078 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:23.332224 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb15a4fc-60ea-4269-8bef-c42fd0372078 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:23.360604 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 563/1134] 10.222.0.26 () {60 vars in 1141 bytes} [Thu Jun 23 01:45:23 2022] POST /identity/v3/projects => generated 311 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:45:25.383051 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ccbb8e-3518-48c9-bb00-b8bdb0ffbb10 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:25.383919 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ccbb8e-3518-48c9-bb00-b8bdb0ffbb10 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:25.383919 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ccbb8e-3518-48c9-bb00-b8bdb0ffbb10 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:25.386165 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 572/1135] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:45:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:45:25.392549 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e071f-98ca-4ca0-955d-6932ecb1af9a None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:25.393007 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e071f-98ca-4ca0-955d-6932ecb1af9a None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:25.393100 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e071f-98ca-4ca0-955d-6932ecb1af9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:25.416027 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-5e6e071f-98ca-4ca0-955d-6932ecb1af9a None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:45:25.417243 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-5e6e071f-98ca-4ca0-955d-6932ecb1af9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:45:25.503431 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 564/1136] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:45:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:45:25.523719 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ccbb8e-3518-48c9-bb00-b8bdb0ffbb10 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:25.567756 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:25.626553 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ccbb8e-3518-48c9-bb00-b8bdb0ffbb10 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:25.662277 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2e0dd-f6c4-406a-b8ee-b79dea68b873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:25.663075 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2e0dd-f6c4-406a-b8ee-b79dea68b873 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:25.663394 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2e0dd-f6c4-406a-b8ee-b79dea68b873 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:25.663669 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2e0dd-f6c4-406a-b8ee-b79dea68b873 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:25.664975 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f2e0dd-f6c4-406a-b8ee-b79dea68b873 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:25.666437 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f2e0dd-f6c4-406a-b8ee-b79dea68b873 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:25.668134 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 573/1137] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:45:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:25.674115 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e071f-98ca-4ca0-955d-6932ecb1af9a None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:25.755240 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:25.776823 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e071f-98ca-4ca0-955d-6932ecb1af9a None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:25.813160 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c8d2a1-d0bc-4841-a855-02f7dccbd33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:25.813823 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8d2a1-d0bc-4841-a855-02f7dccbd33b None admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:25.814101 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8d2a1-d0bc-4841-a855-02f7dccbd33b None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:25.814376 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8d2a1-d0bc-4841-a855-02f7dccbd33b None admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:25.815087 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c8d2a1-d0bc-4841-a855-02f7dccbd33b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:25.816551 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c8d2a1-d0bc-4841-a855-02f7dccbd33b None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:25.842798 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 565/1138] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Jun 23 01:45:25 2022] POST /identity/v3/users => generated 286 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:45:27.881743 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814934c-9b99-495f-99b1-5f4942830398 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:27.882624 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814934c-9b99-495f-99b1-5f4942830398 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:27.882624 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814934c-9b99-495f-99b1-5f4942830398 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:27.884909 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 574/1139] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:45:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:45:27.891223 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e405fa-2659-4bd4-b006-be6676d97441 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:27.891636 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e405fa-2659-4bd4-b006-be6676d97441 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:27.891745 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e405fa-2659-4bd4-b006-be6676d97441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:27.911255 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-11e405fa-2659-4bd4-b006-be6676d97441 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:45:27.912590 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-11e405fa-2659-4bd4-b006-be6676d97441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:45:27.985726 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 566/1140] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:45:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:45:28.021510 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9814934c-9b99-495f-99b1-5f4942830398 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:28.135556 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9814934c-9b99-495f-99b1-5f4942830398 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:28.144557 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:28.173524 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d7fbb-a9c6-4fe1-a4ea-354344db5733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:28.174295 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d7fbb-a9c6-4fe1-a4ea-354344db5733 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:28.174577 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d7fbb-a9c6-4fe1-a4ea-354344db5733 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:28.174839 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d7fbb-a9c6-4fe1-a4ea-354344db5733 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:28.186770 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266d7fbb-a9c6-4fe1-a4ea-354344db5733 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:45:28.194431 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-266d7fbb-a9c6-4fe1-a4ea-354344db5733 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:45:28.194431 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:45:28.194431 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:45:28.194431 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:45:28.194431 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:45:28.194431 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:45:28.194431 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:45:28.194431 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:45:28.194431 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:45:28.194431 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:45:28.194431 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:45:28.194431 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:45:28.194431 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:45:28.194431 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:45:28.194431 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:45:28.194431 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:45:28.195644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:45:28.195644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:45:28.195644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:45:28.195644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:45:28.195644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:45:28.195644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:45:28.195644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:45:28.195644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:45:28.195644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:45:28.195644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:45:28.195644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:45:28.195644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:45:28.195644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:45:28.195644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:45:28.195644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:45:28.195644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:45:28.197262 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:45:28.197262 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:45:28.197262 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:45:28.197262 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:45:28.197262 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:45:28.197262 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:45:28.197262 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:45:28.197262 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:45:28.197262 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:45:28.197262 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:45:28.197262 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:45:28.197262 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:45:28.197262 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 575/1141] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jun 23 01:45:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:45:28.205490 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e405fa-2659-4bd4-b006-be6676d97441 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:28.277660 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:28.315894 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e405fa-2659-4bd4-b006-be6676d97441 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:28.353960 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2ae0a-4ab6-4d0c-ad92-16e0b6604a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:28.354638 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2ae0a-4ab6-4d0c-ad92-16e0b6604a3d None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:28.354919 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2ae0a-4ab6-4d0c-ad92-16e0b6604a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:28.355196 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2ae0a-4ab6-4d0c-ad92-16e0b6604a3d None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:28.355990 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b2ae0a-4ab6-4d0c-ad92-16e0b6604a3d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:45:28.356298 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b2ae0a-4ab6-4d0c-ad92-16e0b6604a3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:28.357684 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b2ae0a-4ab6-4d0c-ad92-16e0b6604a3d None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:28.366442 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 567/1142] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:45:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:28.376183 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d7fbb-a9c6-4fe1-a4ea-354344db5733 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:28.499882 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d7fbb-a9c6-4fe1-a4ea-354344db5733 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:28.541838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653ab2f5-3524-4361-826a-9cc9d242adbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:28.543112 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ab2f5-3524-4361-826a-9cc9d242adbd None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:28.543475 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ab2f5-3524-4361-826a-9cc9d242adbd None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:28.543847 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ab2f5-3524-4361-826a-9cc9d242adbd None admin] PATH_INFO: `/v3/users/64e1e63c15ce46a984925c8017822359` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:28.559445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653ab2f5-3524-4361-826a-9cc9d242adbd None admin] RBAC: Authorizing `identity:get_user(user_id=64e1e63c15ce46a984925c8017822359)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:28.560911 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653ab2f5-3524-4361-826a-9cc9d242adbd None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:28.562661 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 576/1143] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:45:28 2022] GET /identity/v3/users/64e1e63c15ce46a984925c8017822359 => generated 286 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:28.571351 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2ae0a-4ab6-4d0c-ad92-16e0b6604a3d None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:28.680683 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2ae0a-4ab6-4d0c-ad92-16e0b6604a3d None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:28.712795 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:28.726596 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82689b0-9fce-413c-9ff5-f80d9bceabf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:28.727604 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82689b0-9fce-413c-9ff5-f80d9bceabf6 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:28.727971 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82689b0-9fce-413c-9ff5-f80d9bceabf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:28.728355 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82689b0-9fce-413c-9ff5-f80d9bceabf6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:28.802417 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82689b0-9fce-413c-9ff5-f80d9bceabf6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:28.804150 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82689b0-9fce-413c-9ff5-f80d9bceabf6 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:28.905443 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 568/1144] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:45:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 01:45:28.913700 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653ab2f5-3524-4361-826a-9cc9d242adbd None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:28.954204 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:29.042366 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653ab2f5-3524-4361-826a-9cc9d242adbd None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:29.086508 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74686d54-048b-49e6-b6ba-85a215820a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:29.087317 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74686d54-048b-49e6-b6ba-85a215820a49 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:29.087601 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74686d54-048b-49e6-b6ba-85a215820a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:29.087868 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74686d54-048b-49e6-b6ba-85a215820a49 None admin] PATH_INFO: `/v3/projects/829999ad7b82438dbd72123a9401feb1` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:29.089194 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74686d54-048b-49e6-b6ba-85a215820a49 None admin] RBAC: Authorizing `identity:get_project(project_id=829999ad7b82438dbd72123a9401feb1)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:29.090630 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74686d54-048b-49e6-b6ba-85a215820a49 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:29.092361 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 577/1145] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:45:28 2022] GET /identity/v3/projects/829999ad7b82438dbd72123a9401feb1 => generated 311 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:29.099892 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82689b0-9fce-413c-9ff5-f80d9bceabf6 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:29.139124 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:29.231677 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82689b0-9fce-413c-9ff5-f80d9bceabf6 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:29.268155 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e659a6-2984-4487-a3a7-495067c4e235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:29.269072 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e659a6-2984-4487-a3a7-495067c4e235 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:29.269354 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e659a6-2984-4487-a3a7-495067c4e235 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:29.269628 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e659a6-2984-4487-a3a7-495067c4e235 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:29.270852 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e659a6-2984-4487-a3a7-495067c4e235 None admin] RBAC: Adding query filter params (role.id=7b20d65f49ee4a48894b6c060bb5d580, scope.project.id=829999ad7b82438dbd72123a9401feb1, user.id=64e1e63c15ce46a984925c8017822359) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:45:29.271167 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e659a6-2984-4487-a3a7-495067c4e235 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:29.273283 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e659a6-2984-4487-a3a7-495067c4e235 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:29.279190 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 569/1146] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:45:29 2022] GET /identity/v3/role_assignments?user.id=64e1e63c15ce46a984925c8017822359&scope.project.id=829999ad7b82438dbd72123a9401feb1&role.id=7b20d65f49ee4a48894b6c060bb5d580 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:31.282600 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d55fe-a6e4-4927-a197-8f142b9d02e9 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:31.283181 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d55fe-a6e4-4927-a197-8f142b9d02e9 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:31.283181 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d55fe-a6e4-4927-a197-8f142b9d02e9 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:31.284554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 578/1147] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:45:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:45:31.290401 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133ab0b3-7443-48e7-8a44-c51c7a9042d6 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:31.290811 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133ab0b3-7443-48e7-8a44-c51c7a9042d6 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:31.290918 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133ab0b3-7443-48e7-8a44-c51c7a9042d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:31.315989 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-133ab0b3-7443-48e7-8a44-c51c7a9042d6 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:45:31.317179 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-133ab0b3-7443-48e7-8a44-c51c7a9042d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:45:31.388107 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 570/1148] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:45:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:45:31.410730 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5d55fe-a6e4-4927-a197-8f142b9d02e9 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:31.510768 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:31.522446 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5d55fe-a6e4-4927-a197-8f142b9d02e9 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:31.560827 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75774c08-b510-43ac-b9eb-a46328a663c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:31.561532 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75774c08-b510-43ac-b9eb-a46328a663c1 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:31.561809 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75774c08-b510-43ac-b9eb-a46328a663c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:31.562079 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75774c08-b510-43ac-b9eb-a46328a663c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:31.573180 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75774c08-b510-43ac-b9eb-a46328a663c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:45:31.580817 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-75774c08-b510-43ac-b9eb-a46328a663c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:45:31.580817 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:45:31.580817 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:45:31.580817 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:45:31.580817 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:45:31.580817 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:45:31.580817 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:45:31.580817 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:45:31.580817 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:45:31.580817 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:45:31.580817 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:45:31.580817 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:45:31.580817 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:45:31.580817 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:45:31.580817 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:45:31.580817 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:45:31.581880 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:45:31.581880 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:45:31.581880 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:45:31.581880 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:45:31.581880 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:45:31.581880 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:45:31.581880 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:45:31.581880 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:45:31.581880 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:45:31.581880 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:45:31.581880 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:45:31.581880 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:45:31.581880 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:45:31.581880 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:45:31.581880 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:45:31.581880 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:45:31.582738 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:45:31.582738 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:45:31.582738 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:45:31.582738 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:45:31.582738 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:45:31.582738 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:45:31.582738 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:45:31.582738 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:45:31.582738 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:45:31.582738 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:45:31.582738 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:45:31.582738 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:45:31.582738 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 579/1149] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jun 23 01:45:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:45:31.590291 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133ab0b3-7443-48e7-8a44-c51c7a9042d6 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:31.616101 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:31.699362 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133ab0b3-7443-48e7-8a44-c51c7a9042d6 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:31.735633 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2266a531-66a6-48b6-8811-20379157b974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:31.736290 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2266a531-66a6-48b6-8811-20379157b974 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:31.736570 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2266a531-66a6-48b6-8811-20379157b974 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:31.736846 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2266a531-66a6-48b6-8811-20379157b974 None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:31.737595 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2266a531-66a6-48b6-8811-20379157b974 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:45:31.737905 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2266a531-66a6-48b6-8811-20379157b974 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:31.739268 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2266a531-66a6-48b6-8811-20379157b974 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:31.747502 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 571/1150] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:45:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:45:31.753650 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75774c08-b510-43ac-b9eb-a46328a663c1 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:31.862069 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75774c08-b510-43ac-b9eb-a46328a663c1 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:31.896851 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f4305-fd22-4273-bf90-bb83d028cdc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:31.897703 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f4305-fd22-4273-bf90-bb83d028cdc9 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:31.897981 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f4305-fd22-4273-bf90-bb83d028cdc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:31.898251 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f4305-fd22-4273-bf90-bb83d028cdc9 None admin] PATH_INFO: `/v3/users/64e1e63c15ce46a984925c8017822359` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:31.899592 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f4305-fd22-4273-bf90-bb83d028cdc9 None admin] RBAC: Authorizing `identity:get_user(user_id=64e1e63c15ce46a984925c8017822359)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:31.901035 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f4305-fd22-4273-bf90-bb83d028cdc9 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:31.902634 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 580/1151] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:45:31 2022] GET /identity/v3/users/64e1e63c15ce46a984925c8017822359 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:31.908335 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2266a531-66a6-48b6-8811-20379157b974 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:31.997635 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:32.022104 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2266a531-66a6-48b6-8811-20379157b974 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:32.058178 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6046e719-b3b9-43e8-9c82-3e2d2fac1b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:32.059026 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6046e719-b3b9-43e8-9c82-3e2d2fac1b75 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:32.059344 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6046e719-b3b9-43e8-9c82-3e2d2fac1b75 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:32.059615 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6046e719-b3b9-43e8-9c82-3e2d2fac1b75 None admin] PATH_INFO: `/v3/projects/829999ad7b82438dbd72123a9401feb1` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:32.060853 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6046e719-b3b9-43e8-9c82-3e2d2fac1b75 None admin] RBAC: Authorizing `identity:get_project(project_id=829999ad7b82438dbd72123a9401feb1)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:32.062284 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6046e719-b3b9-43e8-9c82-3e2d2fac1b75 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:32.064021 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 572/1152] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:45:31 2022] GET /identity/v3/projects/829999ad7b82438dbd72123a9401feb1 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:32.070556 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f4305-fd22-4273-bf90-bb83d028cdc9 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:32.106583 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:32.186103 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f4305-fd22-4273-bf90-bb83d028cdc9 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:32.222385 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a2300-7ab9-4791-852f-bf337f757420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:32.223260 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a2300-7ab9-4791-852f-bf337f757420 None admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:32.223553 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a2300-7ab9-4791-852f-bf337f757420 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:32.223818 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a2300-7ab9-4791-852f-bf337f757420 None admin] PATH_INFO: `/v3/projects/829999ad7b82438dbd72123a9401feb1/users/64e1e63c15ce46a984925c8017822359/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:32.226027 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5a2300-7ab9-4791-852f-bf337f757420 None admin] RBAC: Authorizing `identity:create_grant(project_id=829999ad7b82438dbd72123a9401feb1, user_id=64e1e63c15ce46a984925c8017822359, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:32.227488 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5a2300-7ab9-4791-852f-bf337f757420 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:32.242780 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 581/1153] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jun 23 01:45:32 2022] PUT /identity/v3/projects/829999ad7b82438dbd72123a9401feb1/users/64e1e63c15ce46a984925c8017822359/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:45:34.221854 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f91e4-d393-42ce-acf0-44e01d4dbd54 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:34.222748 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f91e4-d393-42ce-acf0-44e01d4dbd54 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:34.222748 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f91e4-d393-42ce-acf0-44e01d4dbd54 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:34.225003 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 573/1154] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:45:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:45:34.233026 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac402b0-6dac-4791-98e8-bb1c1aff27d7 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:34.233473 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac402b0-6dac-4791-98e8-bb1c1aff27d7 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:34.233564 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac402b0-6dac-4791-98e8-bb1c1aff27d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:34.252520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-1ac402b0-6dac-4791-98e8-bb1c1aff27d7 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:45:34.253715 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-1ac402b0-6dac-4791-98e8-bb1c1aff27d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:45:34.325083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 582/1155] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:45:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 01:45:34.349107 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f91e4-d393-42ce-acf0-44e01d4dbd54 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:34.458548 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f91e4-d393-42ce-acf0-44e01d4dbd54 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:34.494706 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf43e935-1e40-4fc4-8f11-fb11fcbe8ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:34.495514 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf43e935-1e40-4fc4-8f11-fb11fcbe8ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:34.495794 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf43e935-1e40-4fc4-8f11-fb11fcbe8ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:34.496070 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf43e935-1e40-4fc4-8f11-fb11fcbe8ea1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:34.508556 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf43e935-1e40-4fc4-8f11-fb11fcbe8ea1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:45:34.516165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-cf43e935-1e40-4fc4-8f11-fb11fcbe8ea1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:45:34.516165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:45:34.516165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:45:34.516165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:45:34.516165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:45:34.516165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:45:34.516165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:45:34.516165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:45:34.516165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:45:34.516165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:45:34.516165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:45:34.516165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:45:34.516165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:45:34.516165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:45:34.516165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:45:34.516165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:45:34.517218 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:45:34.517218 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:45:34.517218 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:45:34.517218 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:45:34.517218 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:45:34.517218 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:45:34.517218 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:45:34.517218 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:45:34.517218 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:45:34.517218 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:45:34.517218 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:45:34.517218 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:45:34.517218 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:45:34.517218 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:45:34.517218 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:45:34.517218 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:45:34.518195 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:45:34.518195 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:45:34.518195 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:45:34.518195 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:45:34.518195 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:45:34.518195 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:45:34.518195 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:45:34.518195 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:45:34.518195 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:45:34.518195 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:45:34.518195 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:45:34.518195 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:45:34.518195 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 574/1156] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jun 23 01:45:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:45:34.525268 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac402b0-6dac-4791-98e8-bb1c1aff27d7 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:34.540967 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:34.640655 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac402b0-6dac-4791-98e8-bb1c1aff27d7 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:34.676447 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ea609-4b3a-445b-8f47-7d9b284f5395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:34.677108 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ea609-4b3a-445b-8f47-7d9b284f5395 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:34.677387 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ea609-4b3a-445b-8f47-7d9b284f5395 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:34.677668 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ea609-4b3a-445b-8f47-7d9b284f5395 None admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:34.678783 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791ea609-4b3a-445b-8f47-7d9b284f5395 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:45:34.679092 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791ea609-4b3a-445b-8f47-7d9b284f5395 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:34.680539 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791ea609-4b3a-445b-8f47-7d9b284f5395 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:34.688806 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 583/1157] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:45:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:34.698435 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf43e935-1e40-4fc4-8f11-fb11fcbe8ea1 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:34.716040 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:34.804077 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf43e935-1e40-4fc4-8f11-fb11fcbe8ea1 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:34.842192 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e2ebb6-7367-411a-b62c-e0afc28d21f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:34.843041 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2ebb6-7367-411a-b62c-e0afc28d21f3 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:34.843347 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2ebb6-7367-411a-b62c-e0afc28d21f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:34.843621 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2ebb6-7367-411a-b62c-e0afc28d21f3 None admin] PATH_INFO: `/v3/users/64e1e63c15ce46a984925c8017822359` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:34.844905 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e2ebb6-7367-411a-b62c-e0afc28d21f3 None admin] RBAC: Authorizing `identity:get_user(user_id=64e1e63c15ce46a984925c8017822359)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:34.846338 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e2ebb6-7367-411a-b62c-e0afc28d21f3 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:34.847996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 575/1158] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:45:34 2022] GET /identity/v3/users/64e1e63c15ce46a984925c8017822359 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:34.854689 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ea609-4b3a-445b-8f47-7d9b284f5395 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:34.937342 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:34.970427 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ea609-4b3a-445b-8f47-7d9b284f5395 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:35.011964 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ff986-de01-4597-b1b9-c0f1ca1a1d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:35.012596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ff986-de01-4597-b1b9-c0f1ca1a1d49 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:35.012873 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ff986-de01-4597-b1b9-c0f1ca1a1d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:35.013146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ff986-de01-4597-b1b9-c0f1ca1a1d49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:35.095244 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6ff986-de01-4597-b1b9-c0f1ca1a1d49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:35.096659 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6ff986-de01-4597-b1b9-c0f1ca1a1d49 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:35.178236 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:35.206616 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 584/1159] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:45:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 01:45:35.216228 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e2ebb6-7367-411a-b62c-e0afc28d21f3 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:35.301069 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:35.325236 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e2ebb6-7367-411a-b62c-e0afc28d21f3 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:35.361411 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad474fc4-fbc6-4f69-b952-aa13ebab2cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:35.362206 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad474fc4-fbc6-4f69-b952-aa13ebab2cd6 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:35.362481 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad474fc4-fbc6-4f69-b952-aa13ebab2cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:35.362755 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad474fc4-fbc6-4f69-b952-aa13ebab2cd6 None admin] PATH_INFO: `/v3/projects/829999ad7b82438dbd72123a9401feb1` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:35.364038 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad474fc4-fbc6-4f69-b952-aa13ebab2cd6 None admin] RBAC: Authorizing `identity:get_project(project_id=829999ad7b82438dbd72123a9401feb1)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:35.365475 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad474fc4-fbc6-4f69-b952-aa13ebab2cd6 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:35.367140 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 576/1160] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:45:35 2022] GET /identity/v3/projects/829999ad7b82438dbd72123a9401feb1 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:35.374285 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ff986-de01-4597-b1b9-c0f1ca1a1d49 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:35.497117 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ff986-de01-4597-b1b9-c0f1ca1a1d49 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:35.533055 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e76e2-6bba-4615-9e98-09a6ac4685a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:35.533701 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e76e2-6bba-4615-9e98-09a6ac4685a3 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:35.533983 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e76e2-6bba-4615-9e98-09a6ac4685a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:35.534261 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e76e2-6bba-4615-9e98-09a6ac4685a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:35.535090 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164e76e2-6bba-4615-9e98-09a6ac4685a3 None admin] RBAC: Adding query filter params (role.id=7b20d65f49ee4a48894b6c060bb5d580, scope.project.id=829999ad7b82438dbd72123a9401feb1, user.id=64e1e63c15ce46a984925c8017822359) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:45:35.535434 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164e76e2-6bba-4615-9e98-09a6ac4685a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:35.536894 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164e76e2-6bba-4615-9e98-09a6ac4685a3 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:35.542853 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 585/1161] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:45:35 2022] GET /identity/v3/role_assignments?user.id=64e1e63c15ce46a984925c8017822359&scope.project.id=829999ad7b82438dbd72123a9401feb1&role.id=7b20d65f49ee4a48894b6c060bb5d580 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:37.563418 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161b266-c8b8-470d-836c-70e0008029f6 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:37.564298 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161b266-c8b8-470d-836c-70e0008029f6 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:37.564298 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161b266-c8b8-470d-836c-70e0008029f6 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:37.566399 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 577/1162] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:45:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:45:37.571822 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27503d32-d98c-4f8a-ac87-3ef397b40ef1 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:37.572261 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27503d32-d98c-4f8a-ac87-3ef397b40ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:37.572369 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27503d32-d98c-4f8a-ac87-3ef397b40ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:37.590787 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-27503d32-d98c-4f8a-ac87-3ef397b40ef1 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:45:37.592030 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-27503d32-d98c-4f8a-ac87-3ef397b40ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:45:37.624496 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:37.662959 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 586/1163] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:45:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:45:37.686273 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2161b266-c8b8-470d-836c-70e0008029f6 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:37.790849 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2161b266-c8b8-470d-836c-70e0008029f6 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:37.829256 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e84738-5f6d-4d88-9936-321690f5466e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:37.829828 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e84738-5f6d-4d88-9936-321690f5466e None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:37.830113 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e84738-5f6d-4d88-9936-321690f5466e None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:37.830391 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e84738-5f6d-4d88-9936-321690f5466e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:37.848241 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:37.900643 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e84738-5f6d-4d88-9936-321690f5466e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:37.902102 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e84738-5f6d-4d88-9936-321690f5466e None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:38.006659 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 578/1164] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:45:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 01:45:38.013519 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27503d32-d98c-4f8a-ac87-3ef397b40ef1 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:38.123900 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27503d32-d98c-4f8a-ac87-3ef397b40ef1 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:38.159550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48d3163-9eeb-4f43-8663-02dda29ad97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:38.160334 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d3163-9eeb-4f43-8663-02dda29ad97d None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:38.160612 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d3163-9eeb-4f43-8663-02dda29ad97d None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:38.160882 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d3163-9eeb-4f43-8663-02dda29ad97d None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:38.168015 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48d3163-9eeb-4f43-8663-02dda29ad97d None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:38.169473 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48d3163-9eeb-4f43-8663-02dda29ad97d None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:38.184291 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-b48d3163-9eeb-4f43-8663-02dda29ad97d None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 23 01:45:38.184291 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:45:38.184291 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 23 01:45:38.184291 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 23 01:45:38.184291 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 01:45:38.184291 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 01:45:38.184291 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 01:45:38.184291 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 01:45:38.184291 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 23 01:45:38.184291 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:45:38.184291 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 01:45:38.184291 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:45:38.184291 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:45:38.184291 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:45:38.184291 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:45:38.184291 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:45:38.184291 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:45:38.185475 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:45:38.185475 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:45:38.185475 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:45:38.185475 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:45:38.185475 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:45:38.185475 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:45:38.185475 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:45:38.185475 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:45:38.185475 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:45:38.185475 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:45:38.185475 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:45:38.185475 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:45:38.185475 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:45:38.185475 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:45:38.185475 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:45:38.185475 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:45:38.186611 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:45:38.186611 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:45:38.186611 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:45:38.186611 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:45:38.186611 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:45:38.186611 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:45:38.186611 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:45:38.186611 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:45:38.186611 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:45:38.186611 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:45:38.186611 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 23 01:45:38.186611 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:45:38.186611 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 23 01:45:38.186611 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:45:38.186611 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 587/1165] 10.222.0.26 () {56 vars in 1117 bytes} [Thu Jun 23 01:45:38 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:45:38.191433 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e84738-5f6d-4d88-9936-321690f5466e None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:38.243439 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:38.299942 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e84738-5f6d-4d88-9936-321690f5466e None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:38.337365 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a12eeb-2e52-4fef-9a3a-f8faefdff89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:38.338025 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a12eeb-2e52-4fef-9a3a-f8faefdff89a None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:38.338304 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a12eeb-2e52-4fef-9a3a-f8faefdff89a None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:38.338582 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a12eeb-2e52-4fef-9a3a-f8faefdff89a None admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:38.339333 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a12eeb-2e52-4fef-9a3a-f8faefdff89a None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:45:38.339653 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a12eeb-2e52-4fef-9a3a-f8faefdff89a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:38.341035 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a12eeb-2e52-4fef-9a3a-f8faefdff89a None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:38.348847 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 579/1166] 10.222.0.26 () {56 vars in 1115 bytes} [Thu Jun 23 01:45:38 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:40.340916 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f7177-8037-4d32-bbcf-f693ab523a1f None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:40.341686 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f7177-8037-4d32-bbcf-f693ab523a1f None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:40.341775 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f7177-8037-4d32-bbcf-f693ab523a1f None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:40.343920 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 588/1167] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:45:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:45:40.349339 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee335d66-cdff-4a90-bfde-8eded95b2742 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:40.349777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee335d66-cdff-4a90-bfde-8eded95b2742 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:40.349869 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee335d66-cdff-4a90-bfde-8eded95b2742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:40.368536 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-ee335d66-cdff-4a90-bfde-8eded95b2742 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:45:40.369726 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-ee335d66-cdff-4a90-bfde-8eded95b2742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:45:40.445641 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 580/1168] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:45:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:45:40.477856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f7177-8037-4d32-bbcf-f693ab523a1f None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:40.508440 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:40.587712 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f7177-8037-4d32-bbcf-f693ab523a1f None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:40.622030 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a633d3-14e3-456b-9021-ad2fbbc10767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:40.622665 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a633d3-14e3-456b-9021-ad2fbbc10767 None admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:40.622940 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a633d3-14e3-456b-9021-ad2fbbc10767 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:40.623246 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a633d3-14e3-456b-9021-ad2fbbc10767 None admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:40.623973 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a633d3-14e3-456b-9021-ad2fbbc10767 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:40.625387 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a633d3-14e3-456b-9021-ad2fbbc10767 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:40.647100 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 589/1169] 10.222.0.26 () {60 vars in 1138 bytes} [Thu Jun 23 01:45:40 2022] POST /identity/v3/domains => generated 267 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:45:42.684750 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17875f0-ad50-4da2-96a7-3fb7ef4487d8 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:42.685577 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17875f0-ad50-4da2-96a7-3fb7ef4487d8 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:42.685577 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17875f0-ad50-4da2-96a7-3fb7ef4487d8 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:42.687756 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 581/1170] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:45:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:45:42.693081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110967b-5364-45b0-a5d0-023d9c5968b9 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:42.693504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110967b-5364-45b0-a5d0-023d9c5968b9 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:42.693592 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110967b-5364-45b0-a5d0-023d9c5968b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:42.712479 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-a110967b-5364-45b0-a5d0-023d9c5968b9 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:45:42.713678 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-a110967b-5364-45b0-a5d0-023d9c5968b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:45:42.782476 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 590/1171] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:45:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:45:42.807628 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17875f0-ad50-4da2-96a7-3fb7ef4487d8 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:42.853652 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:42.922968 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17875f0-ad50-4da2-96a7-3fb7ef4487d8 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:42.958505 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e5a92-bb24-4f89-83ce-38cd028c81aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:42.959356 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e5a92-bb24-4f89-83ce-38cd028c81aa None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:42.959633 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e5a92-bb24-4f89-83ce-38cd028c81aa None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:42.959900 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e5a92-bb24-4f89-83ce-38cd028c81aa None admin] PATH_INFO: `/v3/domains/f21cd81a86a54d54bb3c91e64c470748` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:42.970981 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781e5a92-bb24-4f89-83ce-38cd028c81aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=f21cd81a86a54d54bb3c91e64c470748)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:42.972456 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781e5a92-bb24-4f89-83ce-38cd028c81aa None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:42.974075 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 582/1172] 10.222.0.26 () {56 vars in 1183 bytes} [Thu Jun 23 01:45:42 2022] GET /identity/v3/domains/f21cd81a86a54d54bb3c91e64c470748 => generated 267 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:42.980984 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a110967b-5364-45b0-a5d0-023d9c5968b9 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:43.001798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:43.094788 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a110967b-5364-45b0-a5d0-023d9c5968b9 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:43.129922 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da80fd8d-cd56-4ffd-aa9a-c941c1025cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:43.130567 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da80fd8d-cd56-4ffd-aa9a-c941c1025cf2 None admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:43.130847 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da80fd8d-cd56-4ffd-aa9a-c941c1025cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:43.131123 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da80fd8d-cd56-4ffd-aa9a-c941c1025cf2 None admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:43.131859 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da80fd8d-cd56-4ffd-aa9a-c941c1025cf2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:43.133302 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da80fd8d-cd56-4ffd-aa9a-c941c1025cf2 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:43.168785 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 591/1173] 10.222.0.26 () {60 vars in 1141 bytes} [Thu Jun 23 01:45:42 2022] POST /identity/v3/projects => generated 361 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:45:45.206176 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e0603-b5b0-4e51-9a79-7f35f34afe28 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:45.207175 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e0603-b5b0-4e51-9a79-7f35f34afe28 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:45.207175 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e0603-b5b0-4e51-9a79-7f35f34afe28 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:45.209279 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 583/1174] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:45:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:45:45.214591 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dc569b-ebad-4eca-af5d-cbf1f026e512 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:45.215033 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dc569b-ebad-4eca-af5d-cbf1f026e512 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:45.215134 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dc569b-ebad-4eca-af5d-cbf1f026e512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:45.233949 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-b3dc569b-ebad-4eca-af5d-cbf1f026e512 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:45:45.235097 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-b3dc569b-ebad-4eca-af5d-cbf1f026e512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:45:45.305663 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:45.310743 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 592/1175] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:45:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:45:45.335927 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5e0603-b5b0-4e51-9a79-7f35f34afe28 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:45.444503 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5e0603-b5b0-4e51-9a79-7f35f34afe28 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:45.446261 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:45.482920 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b69985-2eb1-474d-a546-52b0fbcec61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:45.483776 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b69985-2eb1-474d-a546-52b0fbcec61d None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:45.484077 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b69985-2eb1-474d-a546-52b0fbcec61d None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:45.484336 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b69985-2eb1-474d-a546-52b0fbcec61d None admin] PATH_INFO: `/v3/domains/f21cd81a86a54d54bb3c91e64c470748` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:45.485548 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b69985-2eb1-474d-a546-52b0fbcec61d None admin] RBAC: Authorizing `identity:get_domain(domain_id=f21cd81a86a54d54bb3c91e64c470748)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:45.486996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b69985-2eb1-474d-a546-52b0fbcec61d None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:45.488634 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 584/1176] 10.222.0.26 () {56 vars in 1183 bytes} [Thu Jun 23 01:45:45 2022] GET /identity/v3/domains/f21cd81a86a54d54bb3c91e64c470748 => generated 267 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:45.494486 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dc569b-ebad-4eca-af5d-cbf1f026e512 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:45.596484 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dc569b-ebad-4eca-af5d-cbf1f026e512 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:45.630709 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cb9af3-376b-4c64-936f-0d9e9cdfe4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:45.631385 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb9af3-376b-4c64-936f-0d9e9cdfe4a9 None admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:45.631664 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb9af3-376b-4c64-936f-0d9e9cdfe4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:45.631939 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb9af3-376b-4c64-936f-0d9e9cdfe4a9 None admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:45.632632 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cb9af3-376b-4c64-936f-0d9e9cdfe4a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:45.634075 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cb9af3-376b-4c64-936f-0d9e9cdfe4a9 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:45.655944 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 593/1177] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Jun 23 01:45:45 2022] POST /identity/v3/users => generated 311 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:45:47.704854 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d02515-d538-4a36-8cd1-e5271a41aa7b None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:47.705707 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d02515-d538-4a36-8cd1-e5271a41aa7b None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:47.705971 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d02515-d538-4a36-8cd1-e5271a41aa7b None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:47.708043 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 585/1178] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:45:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:45:47.716796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc73dab6-d2b9-4b9c-ab32-cc40cedcccc8 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:47.717224 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc73dab6-d2b9-4b9c-ab32-cc40cedcccc8 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:47.717370 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc73dab6-d2b9-4b9c-ab32-cc40cedcccc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:47.736517 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-dc73dab6-d2b9-4b9c-ab32-cc40cedcccc8 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:45:47.737724 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-dc73dab6-d2b9-4b9c-ab32-cc40cedcccc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:45:47.795413 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:47.814406 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 594/1179] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:45:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:45:47.838139 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d02515-d538-4a36-8cd1-e5271a41aa7b None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:47.949431 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d02515-d538-4a36-8cd1-e5271a41aa7b None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:47.990385 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ae0b9e-01a6-43b0-a7a6-d3579fd9241e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:47.991119 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae0b9e-01a6-43b0-a7a6-d3579fd9241e None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:47.991442 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae0b9e-01a6-43b0-a7a6-d3579fd9241e None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:47.991720 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae0b9e-01a6-43b0-a7a6-d3579fd9241e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:48.002617 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ae0b9e-01a6-43b0-a7a6-d3579fd9241e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:45:48.010210 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-44ae0b9e-01a6-43b0-a7a6-d3579fd9241e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:45:48.010210 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:45:48.010210 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:45:48.010210 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:45:48.010210 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:45:48.010210 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:45:48.010210 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:45:48.010210 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:45:48.010210 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:45:48.010210 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:45:48.010210 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:45:48.010210 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:45:48.010210 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:45:48.010210 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:45:48.010210 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:45:48.010210 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:45:48.011431 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:45:48.011431 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:45:48.011431 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:45:48.011431 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:45:48.011431 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:45:48.011431 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:45:48.011431 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:45:48.011431 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:45:48.011431 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:45:48.011431 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:45:48.011431 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:45:48.011431 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:45:48.011431 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:45:48.011431 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:45:48.011431 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:45:48.011431 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:45:48.012774 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:45:48.012774 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:45:48.012774 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:45:48.012774 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:45:48.012774 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:45:48.012774 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:45:48.012774 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:45:48.012774 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:45:48.012774 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:45:48.012774 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:45:48.012774 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:45:48.012774 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:45:48.012774 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 586/1180] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jun 23 01:45:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:45:48.017491 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc73dab6-d2b9-4b9c-ab32-cc40cedcccc8 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:48.124578 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc73dab6-d2b9-4b9c-ab32-cc40cedcccc8 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:48.160923 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041d30a-18b4-4ce6-9079-be28890cad71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:48.161582 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041d30a-18b4-4ce6-9079-be28890cad71 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:48.161859 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041d30a-18b4-4ce6-9079-be28890cad71 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:48.162130 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041d30a-18b4-4ce6-9079-be28890cad71 None admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:48.162885 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a041d30a-18b4-4ce6-9079-be28890cad71 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:45:48.163194 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a041d30a-18b4-4ce6-9079-be28890cad71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:48.164629 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a041d30a-18b4-4ce6-9079-be28890cad71 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:48.174365 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 595/1181] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:45:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:48.180503 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ae0b9e-01a6-43b0-a7a6-d3579fd9241e None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:48.213839 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:48.289077 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ae0b9e-01a6-43b0-a7a6-d3579fd9241e None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:48.330696 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4071f7b9-77cf-445c-adee-412df80cb303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:48.332237 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071f7b9-77cf-445c-adee-412df80cb303 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:48.332736 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071f7b9-77cf-445c-adee-412df80cb303 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:48.333165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071f7b9-77cf-445c-adee-412df80cb303 None admin] PATH_INFO: `/v3/users/f5bbd230e55d4488b200c408505f1b32` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:48.349457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4071f7b9-77cf-445c-adee-412df80cb303 None admin] RBAC: Authorizing `identity:get_user(user_id=f5bbd230e55d4488b200c408505f1b32)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:48.350923 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4071f7b9-77cf-445c-adee-412df80cb303 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:48.352638 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 587/1182] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:45:48 2022] GET /identity/v3/users/f5bbd230e55d4488b200c408505f1b32 => generated 311 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:48.359169 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041d30a-18b4-4ce6-9079-be28890cad71 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:48.417233 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:48.469829 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041d30a-18b4-4ce6-9079-be28890cad71 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:48.504678 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a96a8-f995-4dbb-a9d1-740b5e6abec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:48.505316 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a96a8-f995-4dbb-a9d1-740b5e6abec6 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:48.505599 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a96a8-f995-4dbb-a9d1-740b5e6abec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:48.505877 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a96a8-f995-4dbb-a9d1-740b5e6abec6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:48.574357 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130a96a8-f995-4dbb-a9d1-740b5e6abec6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:48.575781 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130a96a8-f995-4dbb-a9d1-740b5e6abec6 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:48.623455 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:48.679002 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 596/1183] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:45:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 01:45:48.687938 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4071f7b9-77cf-445c-adee-412df80cb303 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:48.781640 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:48.794611 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4071f7b9-77cf-445c-adee-412df80cb303 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:48.830012 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c00fde-40ac-4620-9a55-3fcf52698bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:48.830807 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c00fde-40ac-4620-9a55-3fcf52698bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:48.831088 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c00fde-40ac-4620-9a55-3fcf52698bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:48.831470 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c00fde-40ac-4620-9a55-3fcf52698bf2 None admin] PATH_INFO: `/v3/projects/d9ef9030e98b4f609645eace44ddc3af` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:48.832687 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c00fde-40ac-4620-9a55-3fcf52698bf2 None admin] RBAC: Authorizing `identity:get_project(project_id=d9ef9030e98b4f609645eace44ddc3af)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:48.834134 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c00fde-40ac-4620-9a55-3fcf52698bf2 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:48.835856 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 588/1184] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:45:48 2022] GET /identity/v3/projects/d9ef9030e98b4f609645eace44ddc3af => generated 361 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:48.842798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a96a8-f995-4dbb-a9d1-740b5e6abec6 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:48.953690 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a96a8-f995-4dbb-a9d1-740b5e6abec6 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:48.989350 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9389d2-9522-41f8-ab4a-5e5e103ca19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:48.989984 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9389d2-9522-41f8-ab4a-5e5e103ca19d None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:48.990264 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9389d2-9522-41f8-ab4a-5e5e103ca19d None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:48.990535 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9389d2-9522-41f8-ab4a-5e5e103ca19d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:48.991415 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9389d2-9522-41f8-ab4a-5e5e103ca19d None admin] RBAC: Adding query filter params (role.id=7b20d65f49ee4a48894b6c060bb5d580, scope.project.id=d9ef9030e98b4f609645eace44ddc3af, user.id=f5bbd230e55d4488b200c408505f1b32) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:45:48.991725 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9389d2-9522-41f8-ab4a-5e5e103ca19d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:48.993161 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9389d2-9522-41f8-ab4a-5e5e103ca19d None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:48.998767 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 597/1185] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:45:48 2022] GET /identity/v3/role_assignments?user.id=f5bbd230e55d4488b200c408505f1b32&scope.project.id=d9ef9030e98b4f609645eace44ddc3af&role.id=7b20d65f49ee4a48894b6c060bb5d580 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:51.017334 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f79aa6e-5daa-4b12-9741-f5d043230051 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:51.018219 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f79aa6e-5daa-4b12-9741-f5d043230051 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:51.018219 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f79aa6e-5daa-4b12-9741-f5d043230051 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:51.020489 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 589/1186] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:45:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:45:51.026702 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c06829-a385-4070-8771-58d6b33def26 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:51.027154 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c06829-a385-4070-8771-58d6b33def26 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:51.027361 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c06829-a385-4070-8771-58d6b33def26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:51.052167 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-72c06829-a385-4070-8771-58d6b33def26 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:45:51.053694 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-72c06829-a385-4070-8771-58d6b33def26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:45:51.061178 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:51.127802 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 598/1187] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:45:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:45:51.150941 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f79aa6e-5daa-4b12-9741-f5d043230051 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:51.256646 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f79aa6e-5daa-4b12-9741-f5d043230051 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:51.292516 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f58279d-c1c7-4364-8965-44a389a48558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:51.293239 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58279d-c1c7-4364-8965-44a389a48558 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:51.293518 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58279d-c1c7-4364-8965-44a389a48558 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:51.293792 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58279d-c1c7-4364-8965-44a389a48558 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:51.305315 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f58279d-c1c7-4364-8965-44a389a48558 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:45:51.312979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-7f58279d-c1c7-4364-8965-44a389a48558 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:45:51.312979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:45:51.312979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:45:51.312979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:45:51.312979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:45:51.312979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:45:51.312979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:45:51.312979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:45:51.312979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:45:51.312979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:45:51.312979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:45:51.312979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:45:51.312979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:45:51.312979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:45:51.312979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:45:51.312979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:45:51.314043 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:45:51.314043 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:45:51.314043 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:45:51.314043 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:45:51.314043 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:45:51.314043 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:45:51.314043 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:45:51.314043 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:45:51.314043 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:45:51.314043 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:45:51.314043 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:45:51.314043 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:45:51.314043 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:45:51.314043 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:45:51.314043 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:45:51.314043 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:45:51.315062 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:45:51.315062 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:45:51.315062 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:45:51.315062 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:45:51.315062 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:45:51.315062 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:45:51.315062 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:45:51.315062 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:45:51.315062 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:45:51.315062 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:45:51.315062 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:45:51.315062 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:45:51.315062 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 590/1188] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jun 23 01:45:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:45:51.320560 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c06829-a385-4070-8771-58d6b33def26 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:51.423702 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c06829-a385-4070-8771-58d6b33def26 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:51.460184 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7abb1f-84ff-4524-aa35-38bfac96e9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:51.460849 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7abb1f-84ff-4524-aa35-38bfac96e9e3 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:51.461120 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7abb1f-84ff-4524-aa35-38bfac96e9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:51.461390 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7abb1f-84ff-4524-aa35-38bfac96e9e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:51.462139 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7abb1f-84ff-4524-aa35-38bfac96e9e3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:45:51.462442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7abb1f-84ff-4524-aa35-38bfac96e9e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:51.463878 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7abb1f-84ff-4524-aa35-38bfac96e9e3 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:51.472256 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 599/1189] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:45:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:51.478699 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f58279d-c1c7-4364-8965-44a389a48558 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:51.505656 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:51.602345 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f58279d-c1c7-4364-8965-44a389a48558 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:51.639057 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd6762-2c6b-4ab9-9872-5142f9b02b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:51.639952 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd6762-2c6b-4ab9-9872-5142f9b02b2b None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:51.640253 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd6762-2c6b-4ab9-9872-5142f9b02b2b None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:51.640523 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd6762-2c6b-4ab9-9872-5142f9b02b2b None admin] PATH_INFO: `/v3/users/f5bbd230e55d4488b200c408505f1b32` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:51.641849 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacd6762-2c6b-4ab9-9872-5142f9b02b2b None admin] RBAC: Authorizing `identity:get_user(user_id=f5bbd230e55d4488b200c408505f1b32)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:51.643316 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacd6762-2c6b-4ab9-9872-5142f9b02b2b None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:51.644918 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 591/1190] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:45:51 2022] GET /identity/v3/users/f5bbd230e55d4488b200c408505f1b32 => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:51.653435 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7abb1f-84ff-4524-aa35-38bfac96e9e3 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:51.657850 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:51.766140 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7abb1f-84ff-4524-aa35-38bfac96e9e3 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:51.802434 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1180c325-d013-495e-9cb8-f143868089fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:51.803650 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1180c325-d013-495e-9cb8-f143868089fb None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:51.803928 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1180c325-d013-495e-9cb8-f143868089fb None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:51.804202 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1180c325-d013-495e-9cb8-f143868089fb None admin] PATH_INFO: `/v3/projects/d9ef9030e98b4f609645eace44ddc3af` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:51.805455 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1180c325-d013-495e-9cb8-f143868089fb None admin] RBAC: Authorizing `identity:get_project(project_id=d9ef9030e98b4f609645eace44ddc3af)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:51.806901 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1180c325-d013-495e-9cb8-f143868089fb None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:51.808655 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 600/1191] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:45:51 2022] GET /identity/v3/projects/d9ef9030e98b4f609645eace44ddc3af => generated 361 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:51.818338 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd6762-2c6b-4ab9-9872-5142f9b02b2b None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:51.911283 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:51.928404 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd6762-2c6b-4ab9-9872-5142f9b02b2b None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:51.963239 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5915fe-66bb-41e4-8f6a-b837c68a8094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:51.964123 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5915fe-66bb-41e4-8f6a-b837c68a8094 None admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:51.964404 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5915fe-66bb-41e4-8f6a-b837c68a8094 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:51.964676 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5915fe-66bb-41e4-8f6a-b837c68a8094 None admin] PATH_INFO: `/v3/projects/d9ef9030e98b4f609645eace44ddc3af/users/f5bbd230e55d4488b200c408505f1b32/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:51.966750 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5915fe-66bb-41e4-8f6a-b837c68a8094 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9ef9030e98b4f609645eace44ddc3af, user_id=f5bbd230e55d4488b200c408505f1b32, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:51.968250 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5915fe-66bb-41e4-8f6a-b837c68a8094 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:51.985083 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 592/1192] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jun 23 01:45:51 2022] PUT /identity/v3/projects/d9ef9030e98b4f609645eace44ddc3af/users/f5bbd230e55d4488b200c408505f1b32/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:45:53.971548 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab61ed-21db-4ba1-8f63-6c3a1f51975e None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:53.972420 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab61ed-21db-4ba1-8f63-6c3a1f51975e None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:53.972420 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab61ed-21db-4ba1-8f63-6c3a1f51975e None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:53.974622 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 601/1193] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:45:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:45:53.983538 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4485a80-ee15-4666-b020-7168609d842e None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:53.984015 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4485a80-ee15-4666-b020-7168609d842e None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:53.984237 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4485a80-ee15-4666-b020-7168609d842e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:54.003499 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-c4485a80-ee15-4666-b020-7168609d842e None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:45:54.004775 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-c4485a80-ee15-4666-b020-7168609d842e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:45:54.077244 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 593/1194] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:45:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:45:54.101407 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab61ed-21db-4ba1-8f63-6c3a1f51975e None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:54.183661 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:54.222411 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab61ed-21db-4ba1-8f63-6c3a1f51975e None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:54.256911 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0c9711-a006-45e9-9646-d976becbdacf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:54.257682 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0c9711-a006-45e9-9646-d976becbdacf None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:54.257956 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0c9711-a006-45e9-9646-d976becbdacf None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:54.258228 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0c9711-a006-45e9-9646-d976becbdacf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:54.269944 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0c9711-a006-45e9-9646-d976becbdacf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:45:54.277488 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-9c0c9711-a006-45e9-9646-d976becbdacf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:45:54.277488 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:45:54.277488 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:45:54.277488 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:45:54.277488 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:45:54.277488 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:45:54.277488 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:45:54.277488 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:45:54.277488 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:45:54.277488 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:45:54.277488 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:45:54.277488 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:45:54.277488 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:45:54.277488 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:45:54.277488 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:45:54.277488 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:45:54.278566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:45:54.278566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:45:54.278566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:45:54.278566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:45:54.278566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:45:54.278566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:45:54.278566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:45:54.278566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:45:54.278566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:45:54.278566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:45:54.278566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:45:54.278566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:45:54.278566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:45:54.278566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:45:54.278566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:45:54.278566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:45:54.281084 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:45:54.281084 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:45:54.281084 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:45:54.281084 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:45:54.281084 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:45:54.281084 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:45:54.281084 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:45:54.281084 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:45:54.281084 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:45:54.281084 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:45:54.281084 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:45:54.281084 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:45:54.281084 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 602/1195] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jun 23 01:45:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:45:54.287044 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4485a80-ee15-4666-b020-7168609d842e None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:54.374473 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:54.403885 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4485a80-ee15-4666-b020-7168609d842e None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:54.438288 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968accbb-4450-42ed-a27e-b0095894b0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:54.438950 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968accbb-4450-42ed-a27e-b0095894b0a5 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:54.439254 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968accbb-4450-42ed-a27e-b0095894b0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:54.439533 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968accbb-4450-42ed-a27e-b0095894b0a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:54.440292 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968accbb-4450-42ed-a27e-b0095894b0a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:45:54.440600 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968accbb-4450-42ed-a27e-b0095894b0a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:54.442002 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968accbb-4450-42ed-a27e-b0095894b0a5 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:54.450040 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 594/1196] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:45:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:54.456805 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0c9711-a006-45e9-9646-d976becbdacf None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:54.564775 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0c9711-a006-45e9-9646-d976becbdacf None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:54.595266 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:54.601359 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7d1485-1e69-4cfa-ae26-84cddf907ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:54.602212 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d1485-1e69-4cfa-ae26-84cddf907ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:54.602491 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d1485-1e69-4cfa-ae26-84cddf907ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:54.602755 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d1485-1e69-4cfa-ae26-84cddf907ac0 None admin] PATH_INFO: `/v3/users/f5bbd230e55d4488b200c408505f1b32` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:54.604056 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7d1485-1e69-4cfa-ae26-84cddf907ac0 None admin] RBAC: Authorizing `identity:get_user(user_id=f5bbd230e55d4488b200c408505f1b32)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:54.605511 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7d1485-1e69-4cfa-ae26-84cddf907ac0 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:54.607146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 603/1197] 10.222.0.26 () {56 vars in 1177 bytes} [Thu Jun 23 01:45:54 2022] GET /identity/v3/users/f5bbd230e55d4488b200c408505f1b32 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:54.616515 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968accbb-4450-42ed-a27e-b0095894b0a5 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:54.743067 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968accbb-4450-42ed-a27e-b0095894b0a5 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:54.780389 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7697cf2b-08a0-4823-9498-1cd504e36b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:54.781066 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697cf2b-08a0-4823-9498-1cd504e36b0e None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:54.781341 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697cf2b-08a0-4823-9498-1cd504e36b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:54.781610 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697cf2b-08a0-4823-9498-1cd504e36b0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:54.785807 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:54.853607 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7697cf2b-08a0-4823-9498-1cd504e36b0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:54.854987 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7697cf2b-08a0-4823-9498-1cd504e36b0e None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:54.959961 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 595/1198] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:45:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 01:45:54.967603 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7d1485-1e69-4cfa-ae26-84cddf907ac0 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:55.081474 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7d1485-1e69-4cfa-ae26-84cddf907ac0 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:55.117531 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c9b196-0683-4411-9e1b-a170d79c2994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:55.118306 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9b196-0683-4411-9e1b-a170d79c2994 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:55.118598 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9b196-0683-4411-9e1b-a170d79c2994 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:55.118878 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9b196-0683-4411-9e1b-a170d79c2994 None admin] PATH_INFO: `/v3/projects/d9ef9030e98b4f609645eace44ddc3af` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:55.120150 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c9b196-0683-4411-9e1b-a170d79c2994 None admin] RBAC: Authorizing `identity:get_project(project_id=d9ef9030e98b4f609645eace44ddc3af)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:55.121602 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c9b196-0683-4411-9e1b-a170d79c2994 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:55.123510 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 604/1199] 10.222.0.26 () {56 vars in 1186 bytes} [Thu Jun 23 01:45:54 2022] GET /identity/v3/projects/d9ef9030e98b4f609645eace44ddc3af => generated 361 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:45:55.130581 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7697cf2b-08a0-4823-9498-1cd504e36b0e None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:45:55.174951 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:45:55.239040 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7697cf2b-08a0-4823-9498-1cd504e36b0e None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:45:55.274001 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9615f14-ffdc-4a03-a772-e72f4c2e721d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:45:55.274643 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9615f14-ffdc-4a03-a772-e72f4c2e721d None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:45:55.274926 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9615f14-ffdc-4a03-a772-e72f4c2e721d None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:45:55.275197 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9615f14-ffdc-4a03-a772-e72f4c2e721d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:45:55.276065 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9615f14-ffdc-4a03-a772-e72f4c2e721d None admin] RBAC: Adding query filter params (role.id=7b20d65f49ee4a48894b6c060bb5d580, scope.project.id=d9ef9030e98b4f609645eace44ddc3af, user.id=f5bbd230e55d4488b200c408505f1b32) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:45:55.276372 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9615f14-ffdc-4a03-a772-e72f4c2e721d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:45:55.277811 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9615f14-ffdc-4a03-a772-e72f4c2e721d None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:45:55.284419 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 596/1200] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:45:55 2022] GET /identity/v3/role_assignments?user.id=f5bbd230e55d4488b200c408505f1b32&scope.project.id=d9ef9030e98b4f609645eace44ddc3af&role.id=7b20d65f49ee4a48894b6c060bb5d580 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:46:05.111776 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=66 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:46:05.281241 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=50 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:46:50.194307 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326a7ac-eed5-4b0b-b908-93e73c88f3df None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:46:50.194945 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326a7ac-eed5-4b0b-b908-93e73c88f3df None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:46:50.194945 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326a7ac-eed5-4b0b-b908-93e73c88f3df None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:46:50.196403 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 605/1201] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:46:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:46:50.202187 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53e4bcd-ae09-4d97-a87a-4687b622aee2 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:46:50.202669 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53e4bcd-ae09-4d97-a87a-4687b622aee2 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:46:50.202762 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53e4bcd-ae09-4d97-a87a-4687b622aee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:46:50.221117 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-b53e4bcd-ae09-4d97-a87a-4687b622aee2 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:46:50.222343 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-b53e4bcd-ae09-4d97-a87a-4687b622aee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:46:50.313971 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 597/1202] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:46:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:46:50.340226 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a326a7ac-eed5-4b0b-b908-93e73c88f3df None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:46:50.469644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a326a7ac-eed5-4b0b-b908-93e73c88f3df None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:46:50.515730 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343636b8-d7da-48cf-a371-b72871dc42bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:46:50.516608 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343636b8-d7da-48cf-a371-b72871dc42bf None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:46:50.516886 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343636b8-d7da-48cf-a371-b72871dc42bf None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:46:50.517155 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343636b8-d7da-48cf-a371-b72871dc42bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:46:50.529697 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343636b8-d7da-48cf-a371-b72871dc42bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:46:50.531173 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343636b8-d7da-48cf-a371-b72871dc42bf None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:46:50.557070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-343636b8-d7da-48cf-a371-b72871dc42bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:46:50.557070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:46:50.557070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:46:50.557070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:46:50.557070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:46:50.557070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:46:50.557070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:46:50.557070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:46:50.557070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:46:50.557070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:46:50.557070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:46:50.557070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:46:50.557070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:46:50.557070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:46:50.557070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:46:50.558236 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:46:50.558236 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:46:50.558236 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:46:50.558236 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:46:50.558236 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:46:50.558236 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:46:50.558236 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:46:50.558236 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:46:50.558236 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:46:50.558236 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:46:50.558236 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:46:50.558236 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:46:50.558236 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:46:50.558236 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:46:50.558236 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:46:50.558236 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:46:50.558236 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:46:50.564168 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:46:50.564168 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:46:50.564168 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:46:50.564168 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:46:50.564168 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 606/1203] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:46:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:46:50.564715 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53e4bcd-ae09-4d97-a87a-4687b622aee2 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:46:50.713572 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53e4bcd-ae09-4d97-a87a-4687b622aee2 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:46:50.752077 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:46:50.760040 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2648b68-ed7d-4a8c-a26e-201536dae71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:46:50.761007 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2648b68-ed7d-4a8c-a26e-201536dae71a None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:46:50.761369 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2648b68-ed7d-4a8c-a26e-201536dae71a None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:46:50.761710 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2648b68-ed7d-4a8c-a26e-201536dae71a None admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:46:50.762650 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2648b68-ed7d-4a8c-a26e-201536dae71a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:46:50.762959 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2648b68-ed7d-4a8c-a26e-201536dae71a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:46:50.764749 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2648b68-ed7d-4a8c-a26e-201536dae71a None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:46:50.784816 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 598/1204] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:46:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:46:50.790708 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343636b8-d7da-48cf-a371-b72871dc42bf None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:46:50.881851 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:46:50.958731 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343636b8-d7da-48cf-a371-b72871dc42bf None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:46:51.001672 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2b238f-df4b-4a69-b7b9-8da157e8ede2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:46:51.002400 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b238f-df4b-4a69-b7b9-8da157e8ede2 None admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:46:51.002624 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b238f-df4b-4a69-b7b9-8da157e8ede2 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:46:51.002885 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b238f-df4b-4a69-b7b9-8da157e8ede2 None admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:46:51.003642 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2b238f-df4b-4a69-b7b9-8da157e8ede2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:46:51.005131 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2b238f-df4b-4a69-b7b9-8da157e8ede2 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:46:51.031997 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 607/1205] 10.222.0.26 () {60 vars in 1132 bytes} [Thu Jun 23 01:46:50 2022] POST /identity/v3/users => generated 251 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 01:46:53.617318 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666f1d79-1a05-4692-85c2-19ffa7374108 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:46:53.617870 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666f1d79-1a05-4692-85c2-19ffa7374108 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:46:53.617870 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666f1d79-1a05-4692-85c2-19ffa7374108 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:46:53.619168 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 599/1206] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:46:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:46:53.625111 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d68bf-4e35-42e7-b68a-afef48a662e3 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:46:53.625612 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d68bf-4e35-42e7-b68a-afef48a662e3 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:46:53.625729 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d68bf-4e35-42e7-b68a-afef48a662e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:46:53.643645 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-4e1d68bf-4e35-42e7-b68a-afef48a662e3 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:46:53.644855 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-4e1d68bf-4e35-42e7-b68a-afef48a662e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:46:53.726935 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 608/1207] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:46:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 01:46:53.750406 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666f1d79-1a05-4692-85c2-19ffa7374108 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:46:53.875433 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666f1d79-1a05-4692-85c2-19ffa7374108 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:46:53.918311 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b99ce3-9220-42a3-a9c4-4bc2aecc028c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:46:53.919286 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b99ce3-9220-42a3-a9c4-4bc2aecc028c None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:46:53.919678 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b99ce3-9220-42a3-a9c4-4bc2aecc028c None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:46:53.919985 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b99ce3-9220-42a3-a9c4-4bc2aecc028c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:46:53.932565 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b99ce3-9220-42a3-a9c4-4bc2aecc028c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:46:53.941851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-21b99ce3-9220-42a3-a9c4-4bc2aecc028c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 01:46:53.941851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:46:53.941851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:46:53.941851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:46:53.941851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:46:53.941851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:46:53.941851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:46:53.941851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:46:53.941851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:46:53.941851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:46:53.941851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:46:53.941851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:46:53.941851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:46:53.941851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:46:53.941851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:46:53.941851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:46:53.942930 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:46:53.942930 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:46:53.942930 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:46:53.942930 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:46:53.942930 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:46:53.942930 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:46:53.942930 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:46:53.942930 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:46:53.942930 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:46:53.942930 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:46:53.942930 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:46:53.942930 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:46:53.942930 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:46:53.942930 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:46:53.942930 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:46:53.942930 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:46:53.944196 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:46:53.944196 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:46:53.944196 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:46:53.944196 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:46:53.944196 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:46:53.944196 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:46:53.944196 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:46:53.944196 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:46:53.944196 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:46:53.944196 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:46:53.944196 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 01:46:53.944196 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:46:53.944196 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 600/1208] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jun 23 01:46:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:46:53.949982 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d68bf-4e35-42e7-b68a-afef48a662e3 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:46:54.009358 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:46:54.095616 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d68bf-4e35-42e7-b68a-afef48a662e3 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:46:54.139773 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56db3f0a-6713-4084-b754-ab47aa3713c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:46:54.140378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db3f0a-6713-4084-b754-ab47aa3713c3 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:46:54.140655 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db3f0a-6713-4084-b754-ab47aa3713c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:46:54.140943 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db3f0a-6713-4084-b754-ab47aa3713c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:46:54.141725 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56db3f0a-6713-4084-b754-ab47aa3713c3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:46:54.142032 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56db3f0a-6713-4084-b754-ab47aa3713c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:46:54.143602 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56db3f0a-6713-4084-b754-ab47aa3713c3 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:46:54.156375 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 609/1209] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Jun 23 01:46:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:46:54.163469 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b99ce3-9220-42a3-a9c4-4bc2aecc028c None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:46:54.227087 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:46:54.295394 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b99ce3-9220-42a3-a9c4-4bc2aecc028c None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:46:54.335556 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ba265-d177-4009-8f9d-49456a893c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:46:54.336565 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ba265-d177-4009-8f9d-49456a893c38 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:46:54.336978 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ba265-d177-4009-8f9d-49456a893c38 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:46:54.337385 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ba265-d177-4009-8f9d-49456a893c38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:46:54.351241 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9ba265-d177-4009-8f9d-49456a893c38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:46:54.352868 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9ba265-d177-4009-8f9d-49456a893c38 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:46:54.376133 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-de9ba265-d177-4009-8f9d-49456a893c38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:46:54.376133 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:46:54.376133 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:46:54.376133 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:46:54.376133 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:46:54.376133 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:46:54.376133 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:46:54.376133 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:46:54.376133 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:46:54.376133 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:46:54.376133 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:46:54.376133 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:46:54.376133 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:46:54.376133 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:46:54.376133 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:46:54.377457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:46:54.377457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:46:54.377457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:46:54.377457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:46:54.377457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:46:54.377457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:46:54.377457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:46:54.377457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:46:54.377457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:46:54.377457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:46:54.377457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:46:54.377457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:46:54.377457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:46:54.377457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:46:54.377457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:46:54.377457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:46:54.377457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:46:54.378379 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:46:54.378379 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:46:54.378379 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:46:54.378379 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:46:54.378379 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 601/1210] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:46:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:46:54.384428 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56db3f0a-6713-4084-b754-ab47aa3713c3 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:46:54.506152 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:46:54.509275 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56db3f0a-6713-4084-b754-ab47aa3713c3 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:46:54.555288 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952527dc-081f-43b9-8610-1107f7557e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:46:54.556020 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952527dc-081f-43b9-8610-1107f7557e08 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:46:54.556304 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952527dc-081f-43b9-8610-1107f7557e08 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:46:54.556592 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952527dc-081f-43b9-8610-1107f7557e08 None admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:46:54.557322 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952527dc-081f-43b9-8610-1107f7557e08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:46:54.557644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952527dc-081f-43b9-8610-1107f7557e08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:46:54.559046 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952527dc-081f-43b9-8610-1107f7557e08 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:46:54.571958 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 610/1211] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:46:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:46:54.577935 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ba265-d177-4009-8f9d-49456a893c38 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:46:54.693029 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ba265-d177-4009-8f9d-49456a893c38 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:46:54.694919 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:46:54.731411 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6433d968-abed-4394-a269-2b7cbdeb54dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:46:54.732296 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6433d968-abed-4394-a269-2b7cbdeb54dd None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:46:54.732580 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6433d968-abed-4394-a269-2b7cbdeb54dd None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:46:54.732835 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6433d968-abed-4394-a269-2b7cbdeb54dd None admin] PATH_INFO: `/v3/users/placement` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:46:54.740738 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6433d968-abed-4394-a269-2b7cbdeb54dd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:46:54.742203 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6433d968-abed-4394-a269-2b7cbdeb54dd None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:46:54.757973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-6433d968-abed-4394-a269-2b7cbdeb54dd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 01:46:54.757973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:46:54.757973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:46:54.757973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:46:54.757973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:46:54.757973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:46:54.757973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:46:54.757973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:46:54.757973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:46:54.757973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:46:54.757973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:46:54.757973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:46:54.757973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 01:46:54.757973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 01:46:54.757973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 01:46:54.757973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 01:46:54.759150 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:46:54.759150 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:46:54.759150 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 01:46:54.759150 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:46:54.759150 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 01:46:54.759150 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:46:54.759150 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:46:54.759150 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:46:54.759150 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:46:54.759150 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:46:54.759150 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:46:54.759150 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:46:54.759150 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:46:54.759150 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:46:54.759150 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:46:54.759150 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:46:54.759150 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:46:54.759150 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:46:54.760411 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:46:54.760411 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:46:54.760411 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 01:46:54.760411 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 01:46:54.760411 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 01:46:54.760411 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 01:46:54.760411 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 01:46:54.760411 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 01:46:54.760411 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 01:46:54.760411 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:46:54.760411 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 602/1212] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:46:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:46:54.767275 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952527dc-081f-43b9-8610-1107f7557e08 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:46:54.889206 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952527dc-081f-43b9-8610-1107f7557e08 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:46:54.933703 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b9cd10-6171-44e4-ad60-3cffc34563b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:46:54.934614 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b9cd10-6171-44e4-ad60-3cffc34563b2 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:46:54.934964 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b9cd10-6171-44e4-ad60-3cffc34563b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:46:54.935341 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b9cd10-6171-44e4-ad60-3cffc34563b2 None admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:46:54.936543 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b9cd10-6171-44e4-ad60-3cffc34563b2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:46:54.937053 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b9cd10-6171-44e4-ad60-3cffc34563b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:46:54.938861 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b9cd10-6171-44e4-ad60-3cffc34563b2 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:46:54.959663 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 611/1213] 10.222.0.26 () {56 vars in 1143 bytes} [Thu Jun 23 01:46:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:46:54.965346 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6433d968-abed-4394-a269-2b7cbdeb54dd None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:46:55.086618 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6433d968-abed-4394-a269-2b7cbdeb54dd None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:46:55.131094 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc3d66-013f-489d-b2d6-71823546f97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:46:55.131793 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc3d66-013f-489d-b2d6-71823546f97a None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:46:55.132085 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc3d66-013f-489d-b2d6-71823546f97a None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:46:55.132354 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc3d66-013f-489d-b2d6-71823546f97a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:46:55.169724 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:46:55.215603 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcc3d66-013f-489d-b2d6-71823546f97a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:46:55.217307 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcc3d66-013f-489d-b2d6-71823546f97a None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:46:55.347037 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 603/1214] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:46:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 23 01:46:55.348595 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b9cd10-6171-44e4-ad60-3cffc34563b2 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:46:55.370131 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:46:55.471644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b9cd10-6171-44e4-ad60-3cffc34563b2 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:46:55.506674 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced016e-fe63-4f10-b5f5-3b063303f7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:46:55.507504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced016e-fe63-4f10-b5f5-3b063303f7b0 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:46:55.507783 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced016e-fe63-4f10-b5f5-3b063303f7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:46:55.508075 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced016e-fe63-4f10-b5f5-3b063303f7b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:46:55.520066 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ced016e-fe63-4f10-b5f5-3b063303f7b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:46:55.521529 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ced016e-fe63-4f10-b5f5-3b063303f7b0 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:46:55.539406 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-9ced016e-fe63-4f10-b5f5-3b063303f7b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:46:55.539406 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:46:55.539406 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:46:55.539406 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:46:55.539406 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:46:55.539406 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:46:55.539406 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:46:55.539406 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:46:55.539406 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:46:55.539406 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:46:55.539406 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:46:55.539406 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:46:55.539406 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:46:55.539406 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:46:55.539406 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:46:55.540504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:46:55.540504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:46:55.540504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:46:55.540504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:46:55.540504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:46:55.540504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:46:55.540504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:46:55.540504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:46:55.540504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:46:55.540504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:46:55.540504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:46:55.540504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:46:55.540504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:46:55.540504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:46:55.540504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:46:55.540504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:46:55.540504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:46:55.541632 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:46:55.541632 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:46:55.541632 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:46:55.541632 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:46:55.546089 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 612/1215] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:46:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:46:55.548784 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc3d66-013f-489d-b2d6-71823546f97a None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:46:55.613068 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:46:55.653872 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc3d66-013f-489d-b2d6-71823546f97a None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:46:55.690223 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a5eedc-182d-427d-a794-a93e8b5235a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:46:55.691056 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5eedc-182d-427d-a794-a93e8b5235a0 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:46:55.691464 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5eedc-182d-427d-a794-a93e8b5235a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:46:55.691895 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5eedc-182d-427d-a794-a93e8b5235a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:46:55.692642 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a5eedc-182d-427d-a794-a93e8b5235a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:46:55.692988 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a5eedc-182d-427d-a794-a93e8b5235a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:46:55.694365 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a5eedc-182d-427d-a794-a93e8b5235a0 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:46:55.708334 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 604/1216] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:46:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:46:55.712997 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced016e-fe63-4f10-b5f5-3b063303f7b0 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:46:55.793172 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:46:55.831075 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced016e-fe63-4f10-b5f5-3b063303f7b0 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:46:55.871569 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20ab0b6-b3bb-47b2-8ede-44483e38dc4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:46:55.872297 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ab0b6-b3bb-47b2-8ede-44483e38dc4b None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:46:55.872681 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ab0b6-b3bb-47b2-8ede-44483e38dc4b None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:46:55.872961 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ab0b6-b3bb-47b2-8ede-44483e38dc4b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:46:55.879966 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20ab0b6-b3bb-47b2-8ede-44483e38dc4b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:46:55.881518 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20ab0b6-b3bb-47b2-8ede-44483e38dc4b None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:46:55.895243 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-a20ab0b6-b3bb-47b2-8ede-44483e38dc4b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:46:55.895243 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:46:55.895243 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:46:55.895243 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:46:55.895243 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:46:55.895243 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:46:55.895243 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:46:55.895243 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:46:55.895243 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:46:55.895243 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:46:55.895243 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:46:55.895243 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:46:55.895243 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 01:46:55.895243 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 01:46:55.895243 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 01:46:55.896550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 01:46:55.896550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:46:55.896550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:46:55.896550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:46:55.896550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:46:55.896550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:46:55.896550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:46:55.896550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:46:55.896550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:46:55.896550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:46:55.896550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:46:55.896550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:46:55.896550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:46:55.896550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:46:55.896550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:46:55.896550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:46:55.896550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:46:55.897770 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 01:46:55.897770 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 01:46:55.897770 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 01:46:55.897770 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 01:46:55.897770 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 01:46:55.897770 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 01:46:55.897770 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:46:55.897770 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:46:55.897770 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 613/1217] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:46:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:46:55.902686 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a5eedc-182d-427d-a794-a93e8b5235a0 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:46:56.021952 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a5eedc-182d-427d-a794-a93e8b5235a0 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:46:56.058173 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:46:56.072205 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e603648e-4cbe-4b0f-83f2-f7a2d3113fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:46:56.072883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e603648e-4cbe-4b0f-83f2-f7a2d3113fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:46:56.073168 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e603648e-4cbe-4b0f-83f2-f7a2d3113fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:46:56.073446 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e603648e-4cbe-4b0f-83f2-f7a2d3113fe2 None admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:46:56.074248 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e603648e-4cbe-4b0f-83f2-f7a2d3113fe2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:46:56.074564 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e603648e-4cbe-4b0f-83f2-f7a2d3113fe2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:46:56.076078 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e603648e-4cbe-4b0f-83f2-f7a2d3113fe2 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:46:56.091285 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 605/1218] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jun 23 01:46:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:46:56.096743 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20ab0b6-b3bb-47b2-8ede-44483e38dc4b None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:46:56.223577 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20ab0b6-b3bb-47b2-8ede-44483e38dc4b None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:46:56.256581 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:46:56.263205 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19712f03-b9cf-488d-89be-0765406d479a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:46:56.263946 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19712f03-b9cf-488d-89be-0765406d479a None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:46:56.264237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19712f03-b9cf-488d-89be-0765406d479a None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:46:56.264670 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19712f03-b9cf-488d-89be-0765406d479a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:46:56.265558 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19712f03-b9cf-488d-89be-0765406d479a None admin] RBAC: Adding query filter params (role.id=c20e054f00164967a4c16e7500c93361, scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c, user.id=e5bf3baee1b843fa89b65e5485f6bebb) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:46:56.266288 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19712f03-b9cf-488d-89be-0765406d479a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:46:56.268356 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19712f03-b9cf-488d-89be-0765406d479a None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:46:56.275262 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 614/1219] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:46:56 2022] GET /identity/v3/role_assignments?user.id=e5bf3baee1b843fa89b65e5485f6bebb&scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c&role.id=c20e054f00164967a4c16e7500c93361 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:46:58.432357 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5d6e72-deec-4cf7-872f-4411e9e715a0 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:46:58.432899 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5d6e72-deec-4cf7-872f-4411e9e715a0 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:46:58.432899 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5d6e72-deec-4cf7-872f-4411e9e715a0 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:46:58.434434 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 606/1220] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:46:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:46:58.439864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f0b849-d7d4-4d0c-87c3-40898da4cd1c None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:46:58.440291 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f0b849-d7d4-4d0c-87c3-40898da4cd1c None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:46:58.440381 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f0b849-d7d4-4d0c-87c3-40898da4cd1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:46:58.457932 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-11f0b849-d7d4-4d0c-87c3-40898da4cd1c None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:46:58.459327 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-11f0b849-d7d4-4d0c-87c3-40898da4cd1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:46:58.530758 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 615/1221] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:46:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:46:58.553916 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5d6e72-deec-4cf7-872f-4411e9e715a0 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:46:58.667583 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5d6e72-deec-4cf7-872f-4411e9e715a0 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:46:58.708467 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d08286-9eba-4ff6-871f-a2d62bf95299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:46:58.709389 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d08286-9eba-4ff6-871f-a2d62bf95299 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:46:58.709700 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d08286-9eba-4ff6-871f-a2d62bf95299 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:46:58.709971 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d08286-9eba-4ff6-871f-a2d62bf95299 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:46:58.723131 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d08286-9eba-4ff6-871f-a2d62bf95299 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:46:58.730785 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-e4d08286-9eba-4ff6-871f-a2d62bf95299 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 01:46:58.730785 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:46:58.730785 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:46:58.730785 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:46:58.730785 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:46:58.730785 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:46:58.730785 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:46:58.730785 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:46:58.730785 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:46:58.730785 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:46:58.730785 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:46:58.730785 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:46:58.730785 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:46:58.730785 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:46:58.730785 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:46:58.730785 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:46:58.731959 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:46:58.731959 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:46:58.731959 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:46:58.731959 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:46:58.731959 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:46:58.731959 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:46:58.731959 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:46:58.731959 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:46:58.731959 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:46:58.731959 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:46:58.731959 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:46:58.731959 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:46:58.731959 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:46:58.731959 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:46:58.731959 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:46:58.731959 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:46:58.733148 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:46:58.733148 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:46:58.733148 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:46:58.733148 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:46:58.733148 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:46:58.733148 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:46:58.733148 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:46:58.733148 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:46:58.733148 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:46:58.733148 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:46:58.733148 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 01:46:58.733148 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:46:58.733148 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 607/1222] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jun 23 01:46:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:46:58.738329 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f0b849-d7d4-4d0c-87c3-40898da4cd1c None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:46:58.839367 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f0b849-d7d4-4d0c-87c3-40898da4cd1c None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:46:58.866275 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:46:58.878601 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b798f965-d80b-46b5-8f85-6d6161e726f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:46:58.879302 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798f965-d80b-46b5-8f85-6d6161e726f0 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:46:58.879589 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798f965-d80b-46b5-8f85-6d6161e726f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:46:58.879978 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798f965-d80b-46b5-8f85-6d6161e726f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:46:58.880704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b798f965-d80b-46b5-8f85-6d6161e726f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:46:58.881012 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b798f965-d80b-46b5-8f85-6d6161e726f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:46:58.882404 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b798f965-d80b-46b5-8f85-6d6161e726f0 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:46:58.895752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 616/1223] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Jun 23 01:46:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:46:58.896965 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d08286-9eba-4ff6-871f-a2d62bf95299 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:46:58.946601 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:46:59.008128 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d08286-9eba-4ff6-871f-a2d62bf95299 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:46:59.046287 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da463e80-53d3-4be1-8bf6-3ef990f9df5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:46:59.047324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da463e80-53d3-4be1-8bf6-3ef990f9df5d None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:46:59.047694 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da463e80-53d3-4be1-8bf6-3ef990f9df5d None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:46:59.048004 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da463e80-53d3-4be1-8bf6-3ef990f9df5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:46:59.062789 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da463e80-53d3-4be1-8bf6-3ef990f9df5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:46:59.064487 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da463e80-53d3-4be1-8bf6-3ef990f9df5d None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:46:59.083012 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-da463e80-53d3-4be1-8bf6-3ef990f9df5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:46:59.083012 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:46:59.083012 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:46:59.083012 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:46:59.083012 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:46:59.083012 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:46:59.083012 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:46:59.083012 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:46:59.083012 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:46:59.083012 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:46:59.083012 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:46:59.083012 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:46:59.083012 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:46:59.083012 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:46:59.083012 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:46:59.084132 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:46:59.084132 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:46:59.084132 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:46:59.084132 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:46:59.084132 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:46:59.084132 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:46:59.084132 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:46:59.084132 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:46:59.084132 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:46:59.084132 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:46:59.084132 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:46:59.084132 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:46:59.084132 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:46:59.084132 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:46:59.084132 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:46:59.084132 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:46:59.084132 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:46:59.085081 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:46:59.085081 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:46:59.085081 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:46:59.085081 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:46:59.085081 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 608/1224] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:46:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:46:59.090366 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b798f965-d80b-46b5-8f85-6d6161e726f0 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:46:59.197252 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b798f965-d80b-46b5-8f85-6d6161e726f0 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:46:59.234927 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b8abcd-9ee8-4825-ba04-d7a064e630e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:46:59.235621 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8abcd-9ee8-4825-ba04-d7a064e630e7 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:46:59.235907 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8abcd-9ee8-4825-ba04-d7a064e630e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:46:59.236177 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8abcd-9ee8-4825-ba04-d7a064e630e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:46:59.236903 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b8abcd-9ee8-4825-ba04-d7a064e630e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:46:59.237206 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b8abcd-9ee8-4825-ba04-d7a064e630e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:46:59.238589 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b8abcd-9ee8-4825-ba04-d7a064e630e7 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:46:59.251336 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 617/1225] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:46:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:46:59.257754 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da463e80-53d3-4be1-8bf6-3ef990f9df5d None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:46:59.367103 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:46:59.374748 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da463e80-53d3-4be1-8bf6-3ef990f9df5d None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:46:59.412513 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd5584e-4262-4d06-b015-29818eea6c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:46:59.413398 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5584e-4262-4d06-b015-29818eea6c4a None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:46:59.413677 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5584e-4262-4d06-b015-29818eea6c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:46:59.413951 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5584e-4262-4d06-b015-29818eea6c4a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:46:59.422696 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd5584e-4262-4d06-b015-29818eea6c4a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:46:59.424283 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd5584e-4262-4d06-b015-29818eea6c4a None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:46:59.443647 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-dbd5584e-4262-4d06-b015-29818eea6c4a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 01:46:59.443647 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:46:59.443647 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:46:59.443647 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:46:59.443647 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:46:59.443647 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:46:59.443647 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:46:59.443647 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:46:59.443647 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:46:59.443647 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:46:59.443647 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:46:59.443647 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:46:59.443647 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 01:46:59.443647 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 01:46:59.443647 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 01:46:59.443647 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 01:46:59.450461 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:46:59.450461 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:46:59.450461 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 01:46:59.450461 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:46:59.450461 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 01:46:59.450461 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:46:59.450461 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:46:59.450461 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:46:59.450461 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:46:59.450461 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:46:59.450461 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:46:59.450461 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:46:59.450461 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:46:59.450461 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:46:59.450461 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:46:59.450461 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:46:59.450461 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:46:59.450461 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:46:59.451793 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:46:59.451793 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:46:59.451793 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 01:46:59.451793 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 01:46:59.451793 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 01:46:59.451793 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 01:46:59.451793 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 01:46:59.451793 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 01:46:59.451793 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 01:46:59.451793 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:46:59.451793 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 609/1226] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:46:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:46:59.452499 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b8abcd-9ee8-4825-ba04-d7a064e630e7 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:46:59.501627 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:46:59.566549 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b8abcd-9ee8-4825-ba04-d7a064e630e7 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:46:59.607599 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b13db6-f9a7-4ed8-8d9f-3ff3f5ecdc27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:46:59.608418 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b13db6-f9a7-4ed8-8d9f-3ff3f5ecdc27 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:46:59.608705 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b13db6-f9a7-4ed8-8d9f-3ff3f5ecdc27 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:46:59.608972 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b13db6-f9a7-4ed8-8d9f-3ff3f5ecdc27 None admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:46:59.609849 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b13db6-f9a7-4ed8-8d9f-3ff3f5ecdc27 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:46:59.610285 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b13db6-f9a7-4ed8-8d9f-3ff3f5ecdc27 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:46:59.611887 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b13db6-f9a7-4ed8-8d9f-3ff3f5ecdc27 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:46:59.631250 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 618/1227] 10.222.0.26 () {56 vars in 1143 bytes} [Thu Jun 23 01:46:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:46:59.637010 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd5584e-4262-4d06-b015-29818eea6c4a None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:46:59.750274 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd5584e-4262-4d06-b015-29818eea6c4a None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:46:59.787820 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec7991-9699-41c6-9336-cc0d5841d4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:46:59.788625 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec7991-9699-41c6-9336-cc0d5841d4c0 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:46:59.788903 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec7991-9699-41c6-9336-cc0d5841d4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:46:59.789207 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec7991-9699-41c6-9336-cc0d5841d4c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:46:59.800716 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ec7991-9699-41c6-9336-cc0d5841d4c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:46:59.802564 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ec7991-9699-41c6-9336-cc0d5841d4c0 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:46:59.820764 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-d8ec7991-9699-41c6-9336-cc0d5841d4c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:46:59.820764 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:46:59.820764 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:46:59.820764 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:46:59.820764 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:46:59.820764 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:46:59.820764 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:46:59.820764 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:46:59.820764 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:46:59.820764 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:46:59.820764 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:46:59.820764 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:46:59.820764 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:46:59.820764 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:46:59.820764 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:46:59.821876 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:46:59.821876 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:46:59.821876 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:46:59.821876 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:46:59.821876 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:46:59.821876 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:46:59.821876 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:46:59.821876 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:46:59.821876 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:46:59.821876 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:46:59.821876 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:46:59.821876 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:46:59.821876 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:46:59.821876 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:46:59.821876 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:46:59.821876 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:46:59.821876 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:46:59.822769 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:46:59.822769 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:46:59.822769 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:46:59.822769 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:46:59.822769 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 610/1228] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:46:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:46:59.828255 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b13db6-f9a7-4ed8-8d9f-3ff3f5ecdc27 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:46:59.937888 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b13db6-f9a7-4ed8-8d9f-3ff3f5ecdc27 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:46:59.940571 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:46:59.974159 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551f56eb-cb9e-4b28-ab4d-cd67120e0bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:46:59.974814 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f56eb-cb9e-4b28-ab4d-cd67120e0bb5 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:46:59.975090 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f56eb-cb9e-4b28-ab4d-cd67120e0bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:46:59.975391 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f56eb-cb9e-4b28-ab4d-cd67120e0bb5 None admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:46:59.976085 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551f56eb-cb9e-4b28-ab4d-cd67120e0bb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:46:59.976380 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551f56eb-cb9e-4b28-ab4d-cd67120e0bb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:46:59.977756 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551f56eb-cb9e-4b28-ab4d-cd67120e0bb5 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:46:59.992964 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 619/1229] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:46:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:46:59.999081 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec7991-9699-41c6-9336-cc0d5841d4c0 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:00.009656 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:00.120396 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec7991-9699-41c6-9336-cc0d5841d4c0 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:00.155676 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc12378-e949-446a-862b-2df15e98f8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:00.156823 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc12378-e949-446a-862b-2df15e98f8b5 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:00.157265 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc12378-e949-446a-862b-2df15e98f8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:00.157591 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc12378-e949-446a-862b-2df15e98f8b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:00.165398 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc12378-e949-446a-862b-2df15e98f8b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:00.166900 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc12378-e949-446a-862b-2df15e98f8b5 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:00.182192 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-1bc12378-e949-446a-862b-2df15e98f8b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:47:00.182192 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:47:00.182192 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:47:00.182192 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:47:00.182192 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:47:00.182192 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:47:00.182192 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:47:00.182192 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:47:00.182192 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:47:00.182192 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:47:00.182192 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:47:00.182192 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:47:00.182192 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 01:47:00.182192 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 01:47:00.182192 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 01:47:00.183462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 01:47:00.183462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:47:00.183462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:47:00.183462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:47:00.183462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:47:00.183462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:47:00.183462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:47:00.183462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:47:00.183462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:47:00.183462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:47:00.183462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:47:00.183462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:47:00.183462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:47:00.183462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:47:00.183462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:47:00.183462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:47:00.183462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:47:00.185056 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 01:47:00.185056 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 01:47:00.185056 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 01:47:00.185056 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 01:47:00.185056 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 01:47:00.185056 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 01:47:00.185056 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:47:00.185056 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:47:00.185056 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 611/1230] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:46:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:47:00.192876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551f56eb-cb9e-4b28-ab4d-cd67120e0bb5 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:00.301690 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551f56eb-cb9e-4b28-ab4d-cd67120e0bb5 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:00.338971 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6fb54a-9e91-40f4-9739-2fd95fa5e3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:00.339655 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6fb54a-9e91-40f4-9739-2fd95fa5e3e8 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:00.339934 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6fb54a-9e91-40f4-9739-2fd95fa5e3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:00.340205 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6fb54a-9e91-40f4-9739-2fd95fa5e3e8 None admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:00.341008 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6fb54a-9e91-40f4-9739-2fd95fa5e3e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:47:00.341346 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6fb54a-9e91-40f4-9739-2fd95fa5e3e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:00.342802 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6fb54a-9e91-40f4-9739-2fd95fa5e3e8 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:00.355571 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 620/1231] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jun 23 01:47:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:47:00.362952 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc12378-e949-446a-862b-2df15e98f8b5 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:00.435570 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:00.475370 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc12378-e949-446a-862b-2df15e98f8b5 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:00.515820 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632cd310-945a-4fed-89b3-d889c2c9031b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:00.516736 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632cd310-945a-4fed-89b3-d889c2c9031b None admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:00.517017 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632cd310-945a-4fed-89b3-d889c2c9031b None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:00.517295 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632cd310-945a-4fed-89b3-d889c2c9031b None admin] PATH_INFO: `/v3/projects/a44a5e0e3e2d4b8e9b9cec7b5b5c725c/users/e5bf3baee1b843fa89b65e5485f6bebb/roles/c20e054f00164967a4c16e7500c93361` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:00.535647 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632cd310-945a-4fed-89b3-d889c2c9031b None admin] RBAC: Authorizing `identity:create_grant(project_id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c, user_id=e5bf3baee1b843fa89b65e5485f6bebb, role_id=c20e054f00164967a4c16e7500c93361)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:00.537181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632cd310-945a-4fed-89b3-d889c2c9031b None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:00.553964 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 612/1232] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jun 23 01:47:00 2022] PUT /identity/v3/projects/a44a5e0e3e2d4b8e9b9cec7b5b5c725c/users/e5bf3baee1b843fa89b65e5485f6bebb/roles/c20e054f00164967a4c16e7500c93361 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:47:02.824912 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd05494-0f26-4912-9072-9b88752f691d None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:02.825596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd05494-0f26-4912-9072-9b88752f691d None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:02.825596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd05494-0f26-4912-9072-9b88752f691d None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:02.827126 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 621/1233] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:47:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:47:02.833709 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3044e-221d-4af3-9cd2-6dff48e0bfb8 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:02.834164 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3044e-221d-4af3-9cd2-6dff48e0bfb8 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:02.834348 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3044e-221d-4af3-9cd2-6dff48e0bfb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:02.853564 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-51b3044e-221d-4af3-9cd2-6dff48e0bfb8 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:47:02.854794 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-51b3044e-221d-4af3-9cd2-6dff48e0bfb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:47:02.927296 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 613/1234] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:47:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:47:02.950771 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd05494-0f26-4912-9072-9b88752f691d None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:02.977183 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:03.067611 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd05494-0f26-4912-9072-9b88752f691d None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:03.105703 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d5a6c0-944c-4b96-89aa-290d4fb9ef97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:03.106454 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d5a6c0-944c-4b96-89aa-290d4fb9ef97 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:03.106811 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d5a6c0-944c-4b96-89aa-290d4fb9ef97 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:03.107085 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d5a6c0-944c-4b96-89aa-290d4fb9ef97 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:03.117796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d5a6c0-944c-4b96-89aa-290d4fb9ef97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:47:03.126867 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-d9d5a6c0-944c-4b96-89aa-290d4fb9ef97 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 01:47:03.126867 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:47:03.126867 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:47:03.126867 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:47:03.126867 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:47:03.126867 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:47:03.126867 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:47:03.126867 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:47:03.126867 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:47:03.126867 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:47:03.126867 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:47:03.126867 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:47:03.126867 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:47:03.126867 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:47:03.126867 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:47:03.126867 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:47:03.127863 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:47:03.127863 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:47:03.127863 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:47:03.127863 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:47:03.127863 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:47:03.127863 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:47:03.127863 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:47:03.127863 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:47:03.127863 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:47:03.127863 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:47:03.127863 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:47:03.127863 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:47:03.127863 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:47:03.127863 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:47:03.127863 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:47:03.127863 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:47:03.128704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:47:03.128704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:47:03.128704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:47:03.128704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:47:03.128704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:47:03.128704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:47:03.128704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:47:03.128704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:47:03.128704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:47:03.128704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:47:03.128704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 01:47:03.128704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:47:03.128704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 622/1235] 10.222.0.26 () {56 vars in 1102 bytes} [Thu Jun 23 01:47:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:47:03.134274 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b3044e-221d-4af3-9cd2-6dff48e0bfb8 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:03.184716 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:03.242724 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b3044e-221d-4af3-9cd2-6dff48e0bfb8 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:03.280664 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead50e8-2bd4-4d3b-bccf-5080bc929d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:03.281308 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead50e8-2bd4-4d3b-bccf-5080bc929d32 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:03.281587 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead50e8-2bd4-4d3b-bccf-5080bc929d32 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:03.281858 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead50e8-2bd4-4d3b-bccf-5080bc929d32 None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:03.282606 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ead50e8-2bd4-4d3b-bccf-5080bc929d32 None admin] RBAC: Adding query filter params (name=service) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:47:03.282909 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ead50e8-2bd4-4d3b-bccf-5080bc929d32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:03.284377 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ead50e8-2bd4-4d3b-bccf-5080bc929d32 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:03.297420 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 614/1236] 10.222.0.26 () {56 vars in 1103 bytes} [Thu Jun 23 01:47:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:47:03.298737 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d5a6c0-944c-4b96-89aa-290d4fb9ef97 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:03.386068 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:03.402320 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d5a6c0-944c-4b96-89aa-290d4fb9ef97 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:03.437711 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2858d6f1-2a58-43b7-b3cf-a2e8d09720c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:03.438510 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2858d6f1-2a58-43b7-b3cf-a2e8d09720c2 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:03.438785 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2858d6f1-2a58-43b7-b3cf-a2e8d09720c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:03.439090 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2858d6f1-2a58-43b7-b3cf-a2e8d09720c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:03.450548 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2858d6f1-2a58-43b7-b3cf-a2e8d09720c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:03.452081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2858d6f1-2a58-43b7-b3cf-a2e8d09720c2 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:03.469982 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-2858d6f1-2a58-43b7-b3cf-a2e8d09720c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:47:03.469982 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:47:03.469982 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:47:03.469982 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:47:03.469982 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:47:03.469982 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:47:03.469982 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:47:03.469982 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:47:03.469982 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:47:03.469982 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:47:03.469982 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:47:03.469982 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:47:03.469982 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:47:03.469982 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:47:03.469982 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:47:03.471311 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:47:03.471311 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:47:03.471311 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:47:03.471311 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:47:03.471311 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:47:03.471311 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:47:03.471311 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:47:03.471311 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:47:03.471311 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:47:03.471311 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:47:03.471311 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:47:03.471311 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:47:03.471311 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:47:03.471311 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:47:03.471311 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:47:03.471311 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:47:03.471311 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:47:03.472956 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:47:03.472956 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:47:03.472956 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:47:03.472956 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:47:03.472956 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 623/1237] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:47:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:47:03.477360 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead50e8-2bd4-4d3b-bccf-5080bc929d32 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:03.584728 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead50e8-2bd4-4d3b-bccf-5080bc929d32 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:03.595019 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:03.623684 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc27bd5-00e0-45e4-999f-face355c25a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:03.624603 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc27bd5-00e0-45e4-999f-face355c25a0 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:03.624886 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc27bd5-00e0-45e4-999f-face355c25a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:03.625165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc27bd5-00e0-45e4-999f-face355c25a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:03.625902 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc27bd5-00e0-45e4-999f-face355c25a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:47:03.626211 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc27bd5-00e0-45e4-999f-face355c25a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:03.627647 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc27bd5-00e0-45e4-999f-face355c25a0 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:03.640322 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 615/1238] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:47:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:47:03.646979 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2858d6f1-2a58-43b7-b3cf-a2e8d09720c2 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:03.759716 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2858d6f1-2a58-43b7-b3cf-a2e8d09720c2 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:03.797252 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acb9ec5-128a-450c-bb20-f5c7795b2257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:03.798116 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb9ec5-128a-450c-bb20-f5c7795b2257 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:03.798417 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb9ec5-128a-450c-bb20-f5c7795b2257 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:03.798694 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb9ec5-128a-450c-bb20-f5c7795b2257 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:03.802129 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:03.807699 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acb9ec5-128a-450c-bb20-f5c7795b2257 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:03.809308 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acb9ec5-128a-450c-bb20-f5c7795b2257 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:03.824454 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-0acb9ec5-128a-450c-bb20-f5c7795b2257 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 01:47:03.824454 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:47:03.824454 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:47:03.824454 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:47:03.824454 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:47:03.824454 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:47:03.824454 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:47:03.824454 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:47:03.824454 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:47:03.824454 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:47:03.824454 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:47:03.824454 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:47:03.824454 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 01:47:03.824454 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 01:47:03.824454 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 01:47:03.824454 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 01:47:03.826159 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:47:03.826159 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:47:03.826159 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 01:47:03.826159 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:47:03.826159 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 01:47:03.826159 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:47:03.826159 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:47:03.826159 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:47:03.826159 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:47:03.826159 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:47:03.826159 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:47:03.826159 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:47:03.826159 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:47:03.826159 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:47:03.826159 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:47:03.826159 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:47:03.826159 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:47:03.826159 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:47:03.827482 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:47:03.827482 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:47:03.827482 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 01:47:03.827482 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 01:47:03.827482 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 01:47:03.827482 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 01:47:03.827482 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 01:47:03.827482 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 01:47:03.827482 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 01:47:03.827482 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:47:03.827482 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 624/1239] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:47:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:47:03.831866 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc27bd5-00e0-45e4-999f-face355c25a0 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:03.940223 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc27bd5-00e0-45e4-999f-face355c25a0 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:03.978960 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e408d36b-ec30-4b8f-8445-9f1169db4c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:03.979940 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408d36b-ec30-4b8f-8445-9f1169db4c2c None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:03.980146 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408d36b-ec30-4b8f-8445-9f1169db4c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:03.980427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408d36b-ec30-4b8f-8445-9f1169db4c2c None admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:03.981639 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e408d36b-ec30-4b8f-8445-9f1169db4c2c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:47:03.981957 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e408d36b-ec30-4b8f-8445-9f1169db4c2c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:03.983478 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e408d36b-ec30-4b8f-8445-9f1169db4c2c None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:04.003619 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 616/1240] 10.222.0.26 () {56 vars in 1143 bytes} [Thu Jun 23 01:47:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:47:04.009610 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acb9ec5-128a-450c-bb20-f5c7795b2257 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:04.132221 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acb9ec5-128a-450c-bb20-f5c7795b2257 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:04.168583 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49211e-2b35-48f9-8707-7599d579c208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:04.169258 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49211e-2b35-48f9-8707-7599d579c208 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:04.169539 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49211e-2b35-48f9-8707-7599d579c208 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:04.169812 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49211e-2b35-48f9-8707-7599d579c208 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:04.236318 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:04.244978 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee49211e-2b35-48f9-8707-7599d579c208 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:04.246924 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee49211e-2b35-48f9-8707-7599d579c208 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:04.359174 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 625/1241] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:47:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 01:47:04.365096 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e408d36b-ec30-4b8f-8445-9f1169db4c2c None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:04.394735 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:04.472948 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e408d36b-ec30-4b8f-8445-9f1169db4c2c None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:04.509859 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e875697b-fc1f-48a9-ad49-c4dff7dea1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:04.510603 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e875697b-fc1f-48a9-ad49-c4dff7dea1ea None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:04.510898 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e875697b-fc1f-48a9-ad49-c4dff7dea1ea None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:04.511199 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e875697b-fc1f-48a9-ad49-c4dff7dea1ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:04.521875 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e875697b-fc1f-48a9-ad49-c4dff7dea1ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:04.523516 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e875697b-fc1f-48a9-ad49-c4dff7dea1ea None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:04.541383 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-e875697b-fc1f-48a9-ad49-c4dff7dea1ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:47:04.541383 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:47:04.541383 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:47:04.541383 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:47:04.541383 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:47:04.541383 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:47:04.541383 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:47:04.541383 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:47:04.541383 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:47:04.541383 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:47:04.541383 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:47:04.541383 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:47:04.541383 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:47:04.541383 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:47:04.541383 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:47:04.542510 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:47:04.542510 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:47:04.542510 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:47:04.542510 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:47:04.542510 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:47:04.542510 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:47:04.542510 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:47:04.542510 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:47:04.542510 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:47:04.542510 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:47:04.542510 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:47:04.542510 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:47:04.542510 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:47:04.542510 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:47:04.542510 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:47:04.542510 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:47:04.542510 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:47:04.548254 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:47:04.548254 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:47:04.548254 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:47:04.548254 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:47:04.548254 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 617/1242] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:47:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:47:04.548607 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49211e-2b35-48f9-8707-7599d579c208 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:04.660443 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:04.667262 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49211e-2b35-48f9-8707-7599d579c208 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:04.705358 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a9ae87-3ef9-4d63-a03f-7111e9eb1542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:04.705996 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a9ae87-3ef9-4d63-a03f-7111e9eb1542 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:04.706274 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a9ae87-3ef9-4d63-a03f-7111e9eb1542 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:04.706543 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a9ae87-3ef9-4d63-a03f-7111e9eb1542 None admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:04.707303 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a9ae87-3ef9-4d63-a03f-7111e9eb1542 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:47:04.707646 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a9ae87-3ef9-4d63-a03f-7111e9eb1542 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:04.709083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a9ae87-3ef9-4d63-a03f-7111e9eb1542 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:04.721415 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 626/1243] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:47:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:47:04.727756 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e875697b-fc1f-48a9-ad49-c4dff7dea1ea None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:04.814832 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:04.849512 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e875697b-fc1f-48a9-ad49-c4dff7dea1ea None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:04.890589 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a496932d-f91a-46de-92fe-474fd05f1908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:04.891427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a496932d-f91a-46de-92fe-474fd05f1908 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:04.891706 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a496932d-f91a-46de-92fe-474fd05f1908 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:04.892000 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a496932d-f91a-46de-92fe-474fd05f1908 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:04.901970 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a496932d-f91a-46de-92fe-474fd05f1908 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:04.904295 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a496932d-f91a-46de-92fe-474fd05f1908 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:04.919858 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-a496932d-f91a-46de-92fe-474fd05f1908 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:47:04.919858 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:47:04.919858 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:47:04.919858 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:47:04.919858 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:47:04.919858 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:47:04.919858 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:47:04.919858 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:47:04.919858 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:47:04.919858 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:47:04.919858 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:47:04.919858 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:47:04.919858 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 01:47:04.919858 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 01:47:04.919858 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 01:47:04.920969 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 01:47:04.920969 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:47:04.920969 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:47:04.920969 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:47:04.920969 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:47:04.920969 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:47:04.920969 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:47:04.920969 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:47:04.920969 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:47:04.920969 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:47:04.920969 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:47:04.920969 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:47:04.920969 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:47:04.920969 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:47:04.920969 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:47:04.920969 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:47:04.920969 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:47:04.922019 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 01:47:04.922019 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 01:47:04.922019 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 01:47:04.922019 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 01:47:04.922019 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 01:47:04.922019 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 01:47:04.922019 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:47:04.922019 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:47:04.922019 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 618/1244] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:47:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:47:04.928994 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a9ae87-3ef9-4d63-a03f-7111e9eb1542 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:05.038384 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a9ae87-3ef9-4d63-a03f-7111e9eb1542 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:05.076037 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96441d85-1b68-4861-bdb5-05ab02987c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:05.076686 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96441d85-1b68-4861-bdb5-05ab02987c7d None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:05.077058 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96441d85-1b68-4861-bdb5-05ab02987c7d None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:05.077291 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96441d85-1b68-4861-bdb5-05ab02987c7d None admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:05.078205 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96441d85-1b68-4861-bdb5-05ab02987c7d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:47:05.078513 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96441d85-1b68-4861-bdb5-05ab02987c7d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:05.080167 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96441d85-1b68-4861-bdb5-05ab02987c7d None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:05.093695 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 627/1245] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jun 23 01:47:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:47:05.100134 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a496932d-f91a-46de-92fe-474fd05f1908 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:05.212863 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a496932d-f91a-46de-92fe-474fd05f1908 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:05.249476 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a0833b-3899-4885-9a14-6a9e692c5380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:05.250148 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a0833b-3899-4885-9a14-6a9e692c5380 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:05.250429 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a0833b-3899-4885-9a14-6a9e692c5380 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:05.250706 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a0833b-3899-4885-9a14-6a9e692c5380 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:05.251587 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a0833b-3899-4885-9a14-6a9e692c5380 None admin] RBAC: Adding query filter params (role.id=c20e054f00164967a4c16e7500c93361, scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c, user.id=e5bf3baee1b843fa89b65e5485f6bebb) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:47:05.251900 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a0833b-3899-4885-9a14-6a9e692c5380 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:05.253350 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a0833b-3899-4885-9a14-6a9e692c5380 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:05.259570 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 619/1246] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:47:05 2022] GET /identity/v3/role_assignments?user.id=e5bf3baee1b843fa89b65e5485f6bebb&scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c&role.id=c20e054f00164967a4c16e7500c93361 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:47:07.355057 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238efdff-2d9b-4e8f-892c-398147001b0b None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:07.355819 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238efdff-2d9b-4e8f-892c-398147001b0b None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:07.355819 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238efdff-2d9b-4e8f-892c-398147001b0b None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:07.357094 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 628/1247] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:47:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:47:07.362260 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06853056-6e65-40af-82cc-6fd0bf0841d1 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:07.362829 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06853056-6e65-40af-82cc-6fd0bf0841d1 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:07.362829 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06853056-6e65-40af-82cc-6fd0bf0841d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:07.368162 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:07.382317 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-06853056-6e65-40af-82cc-6fd0bf0841d1 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:47:07.383857 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-06853056-6e65-40af-82cc-6fd0bf0841d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:47:07.465165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 620/1248] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:47:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:47:07.492326 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238efdff-2d9b-4e8f-892c-398147001b0b None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:07.512948 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:07.612450 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238efdff-2d9b-4e8f-892c-398147001b0b None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:07.659338 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1e62ea-47cf-4f42-9c01-e6dbf0a61be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:07.660148 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1e62ea-47cf-4f42-9c01-e6dbf0a61be3 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:07.660429 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1e62ea-47cf-4f42-9c01-e6dbf0a61be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:07.660782 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1e62ea-47cf-4f42-9c01-e6dbf0a61be3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:07.674620 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1e62ea-47cf-4f42-9c01-e6dbf0a61be3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:47:07.684658 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-7f1e62ea-47cf-4f42-9c01-e6dbf0a61be3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:47:07.684658 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:47:07.684658 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:47:07.684658 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:47:07.684658 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:47:07.684658 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:47:07.684658 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:47:07.684658 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:47:07.684658 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:47:07.684658 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:47:07.684658 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:47:07.684658 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:47:07.684658 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:47:07.684658 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:47:07.684658 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:47:07.684658 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:47:07.685818 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:47:07.685818 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:47:07.685818 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:47:07.685818 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:47:07.685818 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:47:07.685818 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:47:07.685818 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:47:07.685818 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:47:07.685818 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:47:07.685818 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:47:07.685818 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:47:07.685818 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:47:07.685818 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:47:07.685818 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:47:07.685818 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:47:07.685818 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:47:07.686784 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:47:07.686784 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:47:07.686784 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:47:07.686784 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:47:07.686784 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:47:07.686784 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:47:07.686784 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:47:07.686784 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:47:07.686784 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:47:07.686784 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:47:07.686784 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:47:07.686784 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:47:07.686784 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 629/1249] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jun 23 01:47:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:47:07.692958 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06853056-6e65-40af-82cc-6fd0bf0841d1 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:07.807188 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06853056-6e65-40af-82cc-6fd0bf0841d1 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:07.838302 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:07.844138 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e388e1-4e1e-4ef8-9dbd-83035d425dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:07.844828 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e388e1-4e1e-4ef8-9dbd-83035d425dfc None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:07.845139 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e388e1-4e1e-4ef8-9dbd-83035d425dfc None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:07.845383 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e388e1-4e1e-4ef8-9dbd-83035d425dfc None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:07.846145 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e388e1-4e1e-4ef8-9dbd-83035d425dfc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:47:07.846451 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e388e1-4e1e-4ef8-9dbd-83035d425dfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:07.847866 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e388e1-4e1e-4ef8-9dbd-83035d425dfc None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:07.857572 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 621/1250] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:47:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:47:07.863785 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1e62ea-47cf-4f42-9c01-e6dbf0a61be3 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:07.961805 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:07.990291 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1e62ea-47cf-4f42-9c01-e6dbf0a61be3 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:08.034757 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509d428-4b11-4be9-b3ab-240fda80f7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:08.035664 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509d428-4b11-4be9-b3ab-240fda80f7a6 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:08.035963 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509d428-4b11-4be9-b3ab-240fda80f7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:08.036301 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509d428-4b11-4be9-b3ab-240fda80f7a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:08.048956 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9509d428-4b11-4be9-b3ab-240fda80f7a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:08.050756 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9509d428-4b11-4be9-b3ab-240fda80f7a6 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:08.069876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-9509d428-4b11-4be9-b3ab-240fda80f7a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:47:08.069876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:47:08.069876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:47:08.069876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:47:08.069876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:47:08.069876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:47:08.069876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:47:08.069876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:47:08.069876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:47:08.069876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:47:08.069876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:47:08.069876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:47:08.069876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:47:08.069876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:47:08.069876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:47:08.071068 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:47:08.071068 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:47:08.071068 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:47:08.071068 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:47:08.071068 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:47:08.071068 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:47:08.071068 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:47:08.071068 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:47:08.071068 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:47:08.071068 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:47:08.071068 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:47:08.071068 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:47:08.071068 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:47:08.071068 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:47:08.071068 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:47:08.071068 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:47:08.071068 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:47:08.072315 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:47:08.072315 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:47:08.072315 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:47:08.072315 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:47:08.072315 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 630/1251] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:47:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:47:08.077114 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e388e1-4e1e-4ef8-9dbd-83035d425dfc None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:08.181269 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e388e1-4e1e-4ef8-9dbd-83035d425dfc None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:08.218024 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd937167-b890-445d-9afb-44f9cd26845d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:08.218704 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd937167-b890-445d-9afb-44f9cd26845d None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:08.218990 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd937167-b890-445d-9afb-44f9cd26845d None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:08.219295 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd937167-b890-445d-9afb-44f9cd26845d None admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:08.220038 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd937167-b890-445d-9afb-44f9cd26845d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:47:08.220371 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd937167-b890-445d-9afb-44f9cd26845d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:08.221738 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd937167-b890-445d-9afb-44f9cd26845d None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:08.234854 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 622/1252] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:47:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:47:08.240654 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509d428-4b11-4be9-b3ab-240fda80f7a6 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:08.352701 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509d428-4b11-4be9-b3ab-240fda80f7a6 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:08.381110 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:08.391035 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8decc86-34fe-4d97-8d48-9652bbcfe6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:08.392190 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8decc86-34fe-4d97-8d48-9652bbcfe6ee None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:08.392654 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8decc86-34fe-4d97-8d48-9652bbcfe6ee None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:08.393778 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8decc86-34fe-4d97-8d48-9652bbcfe6ee None admin] PATH_INFO: `/v3/users/placement` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:08.402819 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8decc86-34fe-4d97-8d48-9652bbcfe6ee None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:08.404354 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8decc86-34fe-4d97-8d48-9652bbcfe6ee None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:08.419377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-a8decc86-34fe-4d97-8d48-9652bbcfe6ee None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 01:47:08.419377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:47:08.419377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:47:08.419377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:47:08.419377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:47:08.419377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:47:08.419377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:47:08.419377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:47:08.419377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:47:08.419377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:47:08.419377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:47:08.419377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:47:08.419377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 01:47:08.419377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 01:47:08.419377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 01:47:08.419377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 01:47:08.420446 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:47:08.420446 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:47:08.420446 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 01:47:08.420446 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:47:08.420446 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 01:47:08.420446 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:47:08.420446 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:47:08.420446 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:47:08.420446 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:47:08.420446 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:47:08.420446 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:47:08.420446 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:47:08.420446 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:47:08.420446 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:47:08.420446 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:47:08.420446 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:47:08.420446 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:47:08.420446 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:47:08.422241 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:47:08.422241 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:47:08.422241 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 01:47:08.422241 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 01:47:08.422241 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 01:47:08.422241 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 01:47:08.422241 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 01:47:08.422241 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 01:47:08.422241 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 01:47:08.422241 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:47:08.422241 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 631/1253] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:47:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:47:08.427115 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd937167-b890-445d-9afb-44f9cd26845d None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:08.488177 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:08.535751 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd937167-b890-445d-9afb-44f9cd26845d None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:08.572174 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70075280-7437-498e-8aca-620acb144c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:08.572863 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70075280-7437-498e-8aca-620acb144c31 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:08.573140 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70075280-7437-498e-8aca-620acb144c31 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:08.573417 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70075280-7437-498e-8aca-620acb144c31 None admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:08.574319 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70075280-7437-498e-8aca-620acb144c31 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:47:08.574600 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70075280-7437-498e-8aca-620acb144c31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:08.576173 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70075280-7437-498e-8aca-620acb144c31 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:08.593602 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 623/1254] 10.222.0.26 () {56 vars in 1143 bytes} [Thu Jun 23 01:47:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:47:08.599738 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8decc86-34fe-4d97-8d48-9652bbcfe6ee None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:08.712244 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8decc86-34fe-4d97-8d48-9652bbcfe6ee None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:08.748288 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5039dd1-1843-497b-a5da-a2e056023103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:08.748876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5039dd1-1843-497b-a5da-a2e056023103 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:08.749158 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5039dd1-1843-497b-a5da-a2e056023103 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:08.749445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5039dd1-1843-497b-a5da-a2e056023103 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:08.809715 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:08.823851 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5039dd1-1843-497b-a5da-a2e056023103 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:08.825348 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5039dd1-1843-497b-a5da-a2e056023103 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:08.934597 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 632/1255] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:47:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 01:47:08.940600 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70075280-7437-498e-8aca-620acb144c31 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:09.056736 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70075280-7437-498e-8aca-620acb144c31 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:09.074610 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:09.101035 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eebcfd-b0fb-47cb-9c81-00ac32c60006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:09.101847 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eebcfd-b0fb-47cb-9c81-00ac32c60006 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:09.102130 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eebcfd-b0fb-47cb-9c81-00ac32c60006 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:09.102406 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eebcfd-b0fb-47cb-9c81-00ac32c60006 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:09.113928 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eebcfd-b0fb-47cb-9c81-00ac32c60006 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:09.115436 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eebcfd-b0fb-47cb-9c81-00ac32c60006 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:09.136511 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-d4eebcfd-b0fb-47cb-9c81-00ac32c60006 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:47:09.136511 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:47:09.136511 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:47:09.136511 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:47:09.136511 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:47:09.136511 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:47:09.136511 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:47:09.136511 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:47:09.136511 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:47:09.136511 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:47:09.136511 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:47:09.136511 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:47:09.136511 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:47:09.136511 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:47:09.136511 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:47:09.137679 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:47:09.137679 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:47:09.137679 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:47:09.137679 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:47:09.137679 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:47:09.137679 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:47:09.137679 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:47:09.137679 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:47:09.137679 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:47:09.137679 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:47:09.137679 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:47:09.137679 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:47:09.137679 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:47:09.137679 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:47:09.137679 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:47:09.137679 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:47:09.137679 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:47:09.138778 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:47:09.138778 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:47:09.138778 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:47:09.138778 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:47:09.138778 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 624/1256] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:47:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:47:09.145289 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5039dd1-1843-497b-a5da-a2e056023103 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:09.241880 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:09.257560 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5039dd1-1843-497b-a5da-a2e056023103 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:09.298867 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21bef65-08ee-421d-a49b-a1aba25d176b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:09.299569 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21bef65-08ee-421d-a49b-a1aba25d176b None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:09.299850 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21bef65-08ee-421d-a49b-a1aba25d176b None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:09.300122 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21bef65-08ee-421d-a49b-a1aba25d176b None admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:09.300860 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21bef65-08ee-421d-a49b-a1aba25d176b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:47:09.301524 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21bef65-08ee-421d-a49b-a1aba25d176b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:09.302961 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21bef65-08ee-421d-a49b-a1aba25d176b None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:09.320948 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 633/1257] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:47:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:47:09.322188 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eebcfd-b0fb-47cb-9c81-00ac32c60006 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:09.435493 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eebcfd-b0fb-47cb-9c81-00ac32c60006 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:09.473600 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed39928b-88f1-4f19-a603-dd608df9ab6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:09.474540 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39928b-88f1-4f19-a603-dd608df9ab6a None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:09.474959 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39928b-88f1-4f19-a603-dd608df9ab6a None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:09.475166 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39928b-88f1-4f19-a603-dd608df9ab6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:09.482504 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed39928b-88f1-4f19-a603-dd608df9ab6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:09.484000 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed39928b-88f1-4f19-a603-dd608df9ab6a None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:09.498324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-ed39928b-88f1-4f19-a603-dd608df9ab6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:47:09.498324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:47:09.498324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:47:09.498324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:47:09.498324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:47:09.498324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:47:09.498324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:47:09.498324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:47:09.498324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:47:09.498324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:47:09.498324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:47:09.498324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:47:09.498324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 01:47:09.498324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 01:47:09.498324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 01:47:09.499494 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 01:47:09.499494 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:47:09.499494 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:47:09.499494 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:47:09.499494 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:47:09.499494 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:47:09.499494 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:47:09.499494 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:47:09.499494 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:47:09.499494 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:47:09.499494 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:47:09.499494 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:47:09.499494 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:47:09.499494 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:47:09.499494 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:47:09.499494 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:47:09.499494 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:47:09.505457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 01:47:09.505457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 01:47:09.505457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 01:47:09.505457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 01:47:09.505457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 01:47:09.505457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 01:47:09.505457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:47:09.505457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:47:09.505457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 625/1258] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:47:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:47:09.505945 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21bef65-08ee-421d-a49b-a1aba25d176b None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:09.610566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21bef65-08ee-421d-a49b-a1aba25d176b None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:09.645569 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbeebea-3e97-4698-a1da-2edf68115290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:09.646225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbeebea-3e97-4698-a1da-2edf68115290 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:09.646510 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbeebea-3e97-4698-a1da-2edf68115290 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:09.646781 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbeebea-3e97-4698-a1da-2edf68115290 None admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:09.647634 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbeebea-3e97-4698-a1da-2edf68115290 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:47:09.647919 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbeebea-3e97-4698-a1da-2edf68115290 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:09.649358 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbeebea-3e97-4698-a1da-2edf68115290 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:09.667773 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 634/1259] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jun 23 01:47:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:47:09.674298 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed39928b-88f1-4f19-a603-dd608df9ab6a None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:09.704013 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:09.786507 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed39928b-88f1-4f19-a603-dd608df9ab6a None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:09.824762 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dbe61b-021a-4e91-8db3-419232eb5ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:09.825521 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dbe61b-021a-4e91-8db3-419232eb5ee8 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:09.825868 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dbe61b-021a-4e91-8db3-419232eb5ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:09.826178 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dbe61b-021a-4e91-8db3-419232eb5ee8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:09.827128 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dbe61b-021a-4e91-8db3-419232eb5ee8 None admin] RBAC: Adding query filter params (role.id=7b20d65f49ee4a48894b6c060bb5d580, scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c, user.id=e5bf3baee1b843fa89b65e5485f6bebb) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:47:09.827472 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dbe61b-021a-4e91-8db3-419232eb5ee8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:09.829220 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dbe61b-021a-4e91-8db3-419232eb5ee8 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:09.835651 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 626/1260] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:47:09 2022] GET /identity/v3/role_assignments?user.id=e5bf3baee1b843fa89b65e5485f6bebb&scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c&role.id=7b20d65f49ee4a48894b6c060bb5d580 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:47:11.855514 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe5fc3-f97f-4470-b3ef-f4328a56d2c6 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:11.856110 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe5fc3-f97f-4470-b3ef-f4328a56d2c6 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:11.856110 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe5fc3-f97f-4470-b3ef-f4328a56d2c6 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:11.857500 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 635/1261] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:47:11.863885 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da797b-5756-427e-af4d-741536cce22c None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:11.864597 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da797b-5756-427e-af4d-741536cce22c None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:11.864728 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da797b-5756-427e-af4d-741536cce22c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:11.884905 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-69da797b-5756-427e-af4d-741536cce22c None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:47:11.886095 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-69da797b-5756-427e-af4d-741536cce22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:47:11.956881 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 627/1262] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:47:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:47:11.979710 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe5fc3-f97f-4470-b3ef-f4328a56d2c6 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:11.991070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:12.099973 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe5fc3-f97f-4470-b3ef-f4328a56d2c6 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:12.139712 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a8edf-bf00-49c0-9a5c-9f77efb443b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:12.140436 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a8edf-bf00-49c0-9a5c-9f77efb443b8 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:12.140712 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a8edf-bf00-49c0-9a5c-9f77efb443b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:12.140981 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a8edf-bf00-49c0-9a5c-9f77efb443b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:12.152275 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843a8edf-bf00-49c0-9a5c-9f77efb443b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:47:12.159907 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-843a8edf-bf00-49c0-9a5c-9f77efb443b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:47:12.159907 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:47:12.159907 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:47:12.159907 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:47:12.159907 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:47:12.159907 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:47:12.159907 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:47:12.159907 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:47:12.159907 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:47:12.159907 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:47:12.159907 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:47:12.159907 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:47:12.159907 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:47:12.159907 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:47:12.159907 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:47:12.159907 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:47:12.160952 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:47:12.160952 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:47:12.160952 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:47:12.160952 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:47:12.160952 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:47:12.160952 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:47:12.160952 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:47:12.160952 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:47:12.160952 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:47:12.160952 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:47:12.160952 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:47:12.160952 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:47:12.160952 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:47:12.160952 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:47:12.160952 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:47:12.160952 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:47:12.161810 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:47:12.161810 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:47:12.161810 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:47:12.161810 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:47:12.161810 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:47:12.161810 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:47:12.161810 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:47:12.161810 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:47:12.161810 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:47:12.161810 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:47:12.161810 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:47:12.161810 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:47:12.161810 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 636/1263] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jun 23 01:47:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:47:12.169567 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69da797b-5756-427e-af4d-741536cce22c None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:12.189876 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:12.280620 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69da797b-5756-427e-af4d-741536cce22c None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:12.317315 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c83e08d-d145-4c87-af1c-a8e6b5f9a4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:12.317994 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83e08d-d145-4c87-af1c-a8e6b5f9a4a3 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:12.318277 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83e08d-d145-4c87-af1c-a8e6b5f9a4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:12.318554 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83e08d-d145-4c87-af1c-a8e6b5f9a4a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:12.319352 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c83e08d-d145-4c87-af1c-a8e6b5f9a4a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:47:12.319663 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c83e08d-d145-4c87-af1c-a8e6b5f9a4a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:12.321053 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c83e08d-d145-4c87-af1c-a8e6b5f9a4a3 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:12.329246 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 628/1264] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:47:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:47:12.338320 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a8edf-bf00-49c0-9a5c-9f77efb443b8 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:12.417739 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:12.452066 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a8edf-bf00-49c0-9a5c-9f77efb443b8 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:12.486749 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c9340f-1ecc-48db-aa21-e0f8cb40e160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:12.487557 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9340f-1ecc-48db-aa21-e0f8cb40e160 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:12.487838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9340f-1ecc-48db-aa21-e0f8cb40e160 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:12.488112 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9340f-1ecc-48db-aa21-e0f8cb40e160 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:12.499100 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c9340f-1ecc-48db-aa21-e0f8cb40e160 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:12.500588 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c9340f-1ecc-48db-aa21-e0f8cb40e160 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:12.517979 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-81c9340f-1ecc-48db-aa21-e0f8cb40e160 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:47:12.517979 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:47:12.517979 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:47:12.517979 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:47:12.517979 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:47:12.517979 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:47:12.517979 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:47:12.517979 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:47:12.517979 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:47:12.517979 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:47:12.517979 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:47:12.517979 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:47:12.517979 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:47:12.517979 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:47:12.517979 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:47:12.519001 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:47:12.519001 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:47:12.519001 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:47:12.519001 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:47:12.519001 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:47:12.519001 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:47:12.519001 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:47:12.519001 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:47:12.519001 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:47:12.519001 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:47:12.519001 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:47:12.519001 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:47:12.519001 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:47:12.519001 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:47:12.519001 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:47:12.519001 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:47:12.519001 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:47:12.520315 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:47:12.520315 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:47:12.520315 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:47:12.520315 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:47:12.520315 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 637/1265] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:47:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:47:12.525756 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c83e08d-d145-4c87-af1c-a8e6b5f9a4a3 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:12.615249 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:12.637830 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c83e08d-d145-4c87-af1c-a8e6b5f9a4a3 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:12.674568 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95093ba4-d958-4801-bf41-dc980eb25321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:12.675155 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95093ba4-d958-4801-bf41-dc980eb25321 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:12.675461 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95093ba4-d958-4801-bf41-dc980eb25321 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:12.675733 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95093ba4-d958-4801-bf41-dc980eb25321 None admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:12.676823 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95093ba4-d958-4801-bf41-dc980eb25321 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:47:12.677127 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95093ba4-d958-4801-bf41-dc980eb25321 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:12.678498 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95093ba4-d958-4801-bf41-dc980eb25321 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:12.690420 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 629/1266] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:47:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:47:12.699693 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c9340f-1ecc-48db-aa21-e0f8cb40e160 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:12.806788 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c9340f-1ecc-48db-aa21-e0f8cb40e160 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:12.834122 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:12.849506 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceae132-3c1e-483d-99cc-d0bf68120ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:12.850306 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceae132-3c1e-483d-99cc-d0bf68120ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:12.850590 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceae132-3c1e-483d-99cc-d0bf68120ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:12.850863 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceae132-3c1e-483d-99cc-d0bf68120ba2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:12.858643 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceae132-3c1e-483d-99cc-d0bf68120ba2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:12.860159 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceae132-3c1e-483d-99cc-d0bf68120ba2 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:12.874596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-7ceae132-3c1e-483d-99cc-d0bf68120ba2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 01:47:12.874596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:47:12.874596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:47:12.874596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:47:12.874596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:47:12.874596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:47:12.874596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:47:12.874596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:47:12.874596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:47:12.874596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:47:12.874596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:47:12.874596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:47:12.874596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 01:47:12.874596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 01:47:12.874596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 01:47:12.874596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 01:47:12.875673 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:47:12.875673 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:47:12.875673 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 01:47:12.875673 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:47:12.875673 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 01:47:12.875673 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:47:12.875673 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:47:12.875673 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:47:12.875673 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:47:12.875673 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:47:12.875673 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:47:12.875673 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:47:12.875673 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:47:12.875673 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:47:12.875673 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:47:12.875673 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:47:12.875673 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:47:12.875673 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:47:12.876663 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:47:12.876663 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:47:12.876663 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 01:47:12.876663 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 01:47:12.876663 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 01:47:12.876663 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 01:47:12.876663 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 01:47:12.876663 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 01:47:12.876663 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 01:47:12.876663 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:47:12.876663 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 638/1267] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:47:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:47:12.881658 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95093ba4-d958-4801-bf41-dc980eb25321 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:12.985352 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95093ba4-d958-4801-bf41-dc980eb25321 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:13.020938 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85767ea-c0f4-4d42-96a1-0cad41ed1400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:13.021605 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85767ea-c0f4-4d42-96a1-0cad41ed1400 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:13.021861 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85767ea-c0f4-4d42-96a1-0cad41ed1400 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:13.022131 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85767ea-c0f4-4d42-96a1-0cad41ed1400 None admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:13.022986 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85767ea-c0f4-4d42-96a1-0cad41ed1400 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:47:13.023308 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85767ea-c0f4-4d42-96a1-0cad41ed1400 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:13.024780 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85767ea-c0f4-4d42-96a1-0cad41ed1400 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:13.036553 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:13.048119 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 630/1268] 10.222.0.26 () {56 vars in 1143 bytes} [Thu Jun 23 01:47:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:47:13.054258 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceae132-3c1e-483d-99cc-d0bf68120ba2 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:13.154940 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceae132-3c1e-483d-99cc-d0bf68120ba2 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:13.190823 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae1660d-8bfb-48ca-ae8a-bf06b2fb5127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:13.191647 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1660d-8bfb-48ca-ae8a-bf06b2fb5127 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:13.191923 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1660d-8bfb-48ca-ae8a-bf06b2fb5127 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:13.192193 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1660d-8bfb-48ca-ae8a-bf06b2fb5127 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:13.203734 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae1660d-8bfb-48ca-ae8a-bf06b2fb5127 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:13.205186 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae1660d-8bfb-48ca-ae8a-bf06b2fb5127 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:13.222704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-dae1660d-8bfb-48ca-ae8a-bf06b2fb5127 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:47:13.222704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:47:13.222704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:47:13.222704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:47:13.222704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:47:13.222704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:47:13.222704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:47:13.222704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:47:13.222704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:47:13.222704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:47:13.222704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:47:13.222704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:47:13.222704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:47:13.222704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:47:13.222704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:47:13.223739 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:47:13.223739 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:47:13.223739 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:47:13.223739 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:47:13.223739 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:47:13.223739 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:47:13.223739 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:47:13.223739 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:47:13.223739 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:47:13.223739 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:47:13.223739 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:47:13.223739 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:47:13.223739 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:47:13.223739 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:47:13.223739 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:47:13.223739 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:47:13.223739 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:47:13.224816 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:47:13.224816 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:47:13.224816 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:47:13.224816 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:47:13.224816 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 639/1269] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:47:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:47:13.232574 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85767ea-c0f4-4d42-96a1-0cad41ed1400 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:13.352142 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85767ea-c0f4-4d42-96a1-0cad41ed1400 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:13.397871 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199f750f-df3c-4640-b635-3d6763746331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:13.398535 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f750f-df3c-4640-b635-3d6763746331 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:13.398820 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f750f-df3c-4640-b635-3d6763746331 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:13.399100 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f750f-df3c-4640-b635-3d6763746331 None admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:13.399864 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199f750f-df3c-4640-b635-3d6763746331 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:47:13.400171 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199f750f-df3c-4640-b635-3d6763746331 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:13.401559 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199f750f-df3c-4640-b635-3d6763746331 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:13.413211 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 631/1270] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:47:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:47:13.419130 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae1660d-8bfb-48ca-ae8a-bf06b2fb5127 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:13.453970 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:13.523362 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae1660d-8bfb-48ca-ae8a-bf06b2fb5127 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:13.558608 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb611cd4-66d9-43f8-a09b-680f83641ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:13.559369 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb611cd4-66d9-43f8-a09b-680f83641ea7 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:13.559651 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb611cd4-66d9-43f8-a09b-680f83641ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:13.559929 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb611cd4-66d9-43f8-a09b-680f83641ea7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:13.566706 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb611cd4-66d9-43f8-a09b-680f83641ea7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:13.568199 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb611cd4-66d9-43f8-a09b-680f83641ea7 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:13.582450 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-cb611cd4-66d9-43f8-a09b-680f83641ea7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:47:13.582450 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:47:13.582450 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:47:13.582450 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:47:13.582450 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:47:13.582450 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:47:13.582450 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:47:13.582450 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:47:13.582450 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:47:13.582450 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:47:13.582450 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:47:13.582450 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:47:13.582450 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 01:47:13.582450 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 01:47:13.582450 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 01:47:13.583520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 01:47:13.583520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:47:13.583520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:47:13.583520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:47:13.583520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:47:13.583520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:47:13.583520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:47:13.583520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:47:13.583520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:47:13.583520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:47:13.583520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:47:13.583520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:47:13.583520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:47:13.583520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:47:13.583520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:47:13.583520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:47:13.583520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:47:13.584659 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 01:47:13.584659 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 01:47:13.584659 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 01:47:13.584659 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 01:47:13.584659 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 01:47:13.584659 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 01:47:13.584659 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:47:13.584659 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:47:13.587578 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 640/1271] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:47:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:47:13.589473 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199f750f-df3c-4640-b635-3d6763746331 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:13.658309 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:13.703020 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199f750f-df3c-4640-b635-3d6763746331 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:13.740594 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1993b26e-0360-46f5-9db9-1899b5c55bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:13.741248 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1993b26e-0360-46f5-9db9-1899b5c55bec None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:13.741520 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1993b26e-0360-46f5-9db9-1899b5c55bec None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:13.741787 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1993b26e-0360-46f5-9db9-1899b5c55bec None admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:13.742577 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1993b26e-0360-46f5-9db9-1899b5c55bec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:47:13.742884 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1993b26e-0360-46f5-9db9-1899b5c55bec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:13.744352 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1993b26e-0360-46f5-9db9-1899b5c55bec None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:13.756495 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 632/1272] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jun 23 01:47:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:47:13.765026 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb611cd4-66d9-43f8-a09b-680f83641ea7 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:13.856247 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:13.865929 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb611cd4-66d9-43f8-a09b-680f83641ea7 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:13.900806 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdeb644-64fd-46e2-bf17-f1c168f3d7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:13.901680 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdeb644-64fd-46e2-bf17-f1c168f3d7c5 None admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:13.901959 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdeb644-64fd-46e2-bf17-f1c168f3d7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:13.902229 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdeb644-64fd-46e2-bf17-f1c168f3d7c5 None admin] PATH_INFO: `/v3/projects/a44a5e0e3e2d4b8e9b9cec7b5b5c725c/users/e5bf3baee1b843fa89b65e5485f6bebb/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:13.904383 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdeb644-64fd-46e2-bf17-f1c168f3d7c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c, user_id=e5bf3baee1b843fa89b65e5485f6bebb, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:13.905833 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdeb644-64fd-46e2-bf17-f1c168f3d7c5 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:13.920730 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 641/1273] 10.222.0.26 () {58 vars in 1439 bytes} [Thu Jun 23 01:47:13 2022] PUT /identity/v3/projects/a44a5e0e3e2d4b8e9b9cec7b5b5c725c/users/e5bf3baee1b843fa89b65e5485f6bebb/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:47:15.872053 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14236948-e3bb-4f5b-a3f8-dfbc25ec6fc4 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:15.872954 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14236948-e3bb-4f5b-a3f8-dfbc25ec6fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:15.872954 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14236948-e3bb-4f5b-a3f8-dfbc25ec6fc4 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:15.875146 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 633/1274] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:47:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:47:15.881322 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03735cf9-28bb-4dfe-b748-6d95b396aa70 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:15.881748 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03735cf9-28bb-4dfe-b748-6d95b396aa70 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:15.881921 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03735cf9-28bb-4dfe-b748-6d95b396aa70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:15.902493 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-03735cf9-28bb-4dfe-b748-6d95b396aa70 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:47:15.903727 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-03735cf9-28bb-4dfe-b748-6d95b396aa70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:47:15.974668 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 642/1275] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:47:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:47:15.998798 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14236948-e3bb-4f5b-a3f8-dfbc25ec6fc4 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:16.114291 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14236948-e3bb-4f5b-a3f8-dfbc25ec6fc4 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:16.131993 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:16.152297 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f3c195-974d-438a-a7fe-6ddf87968e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:16.153135 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f3c195-974d-438a-a7fe-6ddf87968e02 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:16.153412 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f3c195-974d-438a-a7fe-6ddf87968e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:16.153686 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f3c195-974d-438a-a7fe-6ddf87968e02 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:16.164450 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f3c195-974d-438a-a7fe-6ddf87968e02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 01:47:16.172047 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-b9f3c195-974d-438a-a7fe-6ddf87968e02 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:47:16.172047 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:47:16.172047 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:47:16.172047 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:47:16.172047 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:47:16.172047 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:47:16.172047 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:47:16.172047 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:47:16.172047 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:47:16.172047 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:47:16.172047 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:47:16.172047 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:47:16.172047 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 01:47:16.172047 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 01:47:16.172047 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 01:47:16.172047 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 01:47:16.173022 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 01:47:16.173022 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 01:47:16.173022 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 01:47:16.173022 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 01:47:16.173022 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:47:16.173022 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:47:16.173022 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:47:16.173022 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:47:16.173022 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:47:16.173022 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:47:16.173022 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:47:16.173022 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:47:16.173022 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:47:16.173022 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:47:16.173022 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:47:16.173022 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:47:16.173882 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:47:16.173882 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:47:16.173882 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:47:16.173882 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:47:16.173882 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 01:47:16.173882 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 01:47:16.173882 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 01:47:16.173882 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 01:47:16.173882 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 01:47:16.173882 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 01:47:16.173882 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 01:47:16.173882 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:47:16.173882 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 634/1276] 10.222.0.26 () {56 vars in 1096 bytes} [Thu Jun 23 01:47:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:47:16.179907 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03735cf9-28bb-4dfe-b748-6d95b396aa70 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:16.261442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:16.285918 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03735cf9-28bb-4dfe-b748-6d95b396aa70 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:16.319030 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d357b-4256-4316-b845-2568c1979e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:16.319716 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d357b-4256-4316-b845-2568c1979e05 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:16.319993 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d357b-4256-4316-b845-2568c1979e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:16.320259 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d357b-4256-4316-b845-2568c1979e05 None admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:16.320999 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695d357b-4256-4316-b845-2568c1979e05 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:47:16.321298 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695d357b-4256-4316-b845-2568c1979e05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:16.322671 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695d357b-4256-4316-b845-2568c1979e05 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:16.330475 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 643/1277] 10.222.0.26 () {56 vars in 1099 bytes} [Thu Jun 23 01:47:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:47:16.336305 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f3c195-974d-438a-a7fe-6ddf87968e02 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:16.443956 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f3c195-974d-438a-a7fe-6ddf87968e02 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:16.483266 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02dfe8a-9580-4e3a-9493-895dad387bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:16.484110 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02dfe8a-9580-4e3a-9493-895dad387bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:16.484390 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02dfe8a-9580-4e3a-9493-895dad387bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:16.484661 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02dfe8a-9580-4e3a-9493-895dad387bd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:16.496840 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02dfe8a-9580-4e3a-9493-895dad387bd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:16.498325 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02dfe8a-9580-4e3a-9493-895dad387bd8 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:16.517230 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-e02dfe8a-9580-4e3a-9493-895dad387bd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:47:16.517230 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:47:16.517230 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:47:16.517230 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:47:16.517230 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:47:16.517230 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:47:16.517230 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:47:16.517230 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:47:16.517230 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:47:16.517230 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:47:16.517230 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:47:16.517230 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:47:16.517230 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:47:16.517230 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:47:16.517230 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:47:16.518223 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:47:16.518223 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:47:16.518223 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:47:16.518223 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:47:16.518223 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:47:16.518223 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:47:16.518223 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:47:16.518223 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:47:16.518223 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:47:16.518223 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:47:16.518223 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:47:16.518223 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:47:16.518223 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:47:16.518223 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:47:16.518223 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:47:16.518223 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:47:16.518223 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:47:16.519197 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:47:16.519197 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:47:16.519197 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:47:16.519197 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:47:16.519197 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 635/1278] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:47:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:47:16.524508 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d357b-4256-4316-b845-2568c1979e05 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:16.627899 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d357b-4256-4316-b845-2568c1979e05 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:16.662016 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5263b3f-ec85-4e1f-b831-bd09b63004be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:16.662654 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5263b3f-ec85-4e1f-b831-bd09b63004be None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:16.662929 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5263b3f-ec85-4e1f-b831-bd09b63004be None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:16.663196 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5263b3f-ec85-4e1f-b831-bd09b63004be None admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:16.663951 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5263b3f-ec85-4e1f-b831-bd09b63004be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:47:16.664253 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5263b3f-ec85-4e1f-b831-bd09b63004be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:16.665644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5263b3f-ec85-4e1f-b831-bd09b63004be None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:16.677783 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 644/1279] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:47:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:47:16.683592 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02dfe8a-9580-4e3a-9493-895dad387bd8 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:16.715483 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:16.786961 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02dfe8a-9580-4e3a-9493-895dad387bd8 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:16.828341 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacc0328-b6f5-4a45-9047-9953edd81136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:16.829136 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc0328-b6f5-4a45-9047-9953edd81136 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:16.829412 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc0328-b6f5-4a45-9047-9953edd81136 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:16.829682 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc0328-b6f5-4a45-9047-9953edd81136 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:16.839807 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacc0328-b6f5-4a45-9047-9953edd81136 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:16.841324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacc0328-b6f5-4a45-9047-9953edd81136 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:16.855435 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-bacc0328-b6f5-4a45-9047-9953edd81136 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 01:47:16.855435 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:47:16.855435 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:47:16.855435 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:47:16.855435 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:47:16.855435 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:47:16.855435 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:47:16.855435 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:47:16.855435 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:47:16.855435 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:47:16.855435 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:47:16.855435 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:47:16.855435 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 01:47:16.855435 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 01:47:16.855435 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 01:47:16.855435 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 01:47:16.856490 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:47:16.856490 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:47:16.856490 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 01:47:16.856490 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:47:16.856490 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 01:47:16.856490 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 01:47:16.856490 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:47:16.856490 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:47:16.856490 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:47:16.856490 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:47:16.856490 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:47:16.856490 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:47:16.856490 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:47:16.856490 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:47:16.856490 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:47:16.856490 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:47:16.856490 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:47:16.856490 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:47:16.857593 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:47:16.857593 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:47:16.857593 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 01:47:16.857593 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 01:47:16.857593 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 01:47:16.857593 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 01:47:16.857593 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 01:47:16.857593 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 01:47:16.857593 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 01:47:16.857593 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:47:16.857593 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 636/1280] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:47:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:47:16.862653 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5263b3f-ec85-4e1f-b831-bd09b63004be None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:16.866941 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:16.967309 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5263b3f-ec85-4e1f-b831-bd09b63004be None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:17.002741 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15620f76-b2e4-42a4-90f0-b0398363da91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:17.003413 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15620f76-b2e4-42a4-90f0-b0398363da91 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:17.003689 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15620f76-b2e4-42a4-90f0-b0398363da91 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:17.003956 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15620f76-b2e4-42a4-90f0-b0398363da91 None admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:17.004793 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15620f76-b2e4-42a4-90f0-b0398363da91 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:47:17.005099 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15620f76-b2e4-42a4-90f0-b0398363da91 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:17.006538 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15620f76-b2e4-42a4-90f0-b0398363da91 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:17.023499 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 645/1281] 10.222.0.26 () {56 vars in 1143 bytes} [Thu Jun 23 01:47:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:47:17.029503 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacc0328-b6f5-4a45-9047-9953edd81136 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:17.134239 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:17.147841 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacc0328-b6f5-4a45-9047-9953edd81136 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:17.190222 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b738bc-274e-443f-b65e-29d7b9600fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:17.191198 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b738bc-274e-443f-b65e-29d7b9600fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:17.191668 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b738bc-274e-443f-b65e-29d7b9600fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:17.192097 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b738bc-274e-443f-b65e-29d7b9600fb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:17.270683 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b738bc-274e-443f-b65e-29d7b9600fb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:17.272124 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b738bc-274e-443f-b65e-29d7b9600fb8 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:17.359566 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:17.370026 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 637/1282] 10.222.0.26 () {58 vars in 1282 bytes} [Thu Jun 23 01:47:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 01:47:17.376067 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15620f76-b2e4-42a4-90f0-b0398363da91 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:17.439519 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:17.480591 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15620f76-b2e4-42a4-90f0-b0398363da91 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:17.515247 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1f161-a8c3-4e86-bc66-c68cf6a09e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:17.516050 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1f161-a8c3-4e86-bc66-c68cf6a09e5b None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:17.516323 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1f161-a8c3-4e86-bc66-c68cf6a09e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:17.516596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1f161-a8c3-4e86-bc66-c68cf6a09e5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:17.526968 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e1f161-a8c3-4e86-bc66-c68cf6a09e5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:17.528395 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e1f161-a8c3-4e86-bc66-c68cf6a09e5b None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:17.545935 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-62e1f161-a8c3-4e86-bc66-c68cf6a09e5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:47:17.545935 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:47:17.545935 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:47:17.545935 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:47:17.545935 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:47:17.545935 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:47:17.545935 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:47:17.545935 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:47:17.545935 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:47:17.545935 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:47:17.545935 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:47:17.545935 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:47:17.545935 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 01:47:17.545935 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 01:47:17.545935 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 01:47:17.546995 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 01:47:17.546995 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:47:17.546995 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:47:17.546995 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:47:17.546995 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:47:17.546995 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:47:17.546995 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:47:17.546995 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:47:17.546995 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:47:17.546995 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:47:17.546995 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:47:17.546995 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:47:17.546995 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:47:17.546995 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:47:17.546995 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:47:17.546995 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:47:17.546995 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:47:17.548028 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 01:47:17.548028 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 01:47:17.548028 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 01:47:17.548028 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:47:17.548028 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 646/1283] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:47:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:47:17.555321 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b738bc-274e-443f-b65e-29d7b9600fb8 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:17.655822 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b738bc-274e-443f-b65e-29d7b9600fb8 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:17.691416 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde314aa-333a-4ae1-b3d8-d6249567377e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:17.692067 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde314aa-333a-4ae1-b3d8-d6249567377e None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:17.692341 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde314aa-333a-4ae1-b3d8-d6249567377e None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:17.692610 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde314aa-333a-4ae1-b3d8-d6249567377e None admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:17.693325 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde314aa-333a-4ae1-b3d8-d6249567377e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:47:17.693625 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde314aa-333a-4ae1-b3d8-d6249567377e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:17.694993 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde314aa-333a-4ae1-b3d8-d6249567377e None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:17.706525 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 638/1284] 10.222.0.26 () {56 vars in 1109 bytes} [Thu Jun 23 01:47:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:47:17.712456 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1f161-a8c3-4e86-bc66-c68cf6a09e5b None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:17.822720 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1f161-a8c3-4e86-bc66-c68cf6a09e5b None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:17.838270 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:17.860627 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a7e387-b5b4-4826-9a9b-a5b43431a8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:17.861401 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7e387-b5b4-4826-9a9b-a5b43431a8bd None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:17.861675 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7e387-b5b4-4826-9a9b-a5b43431a8bd None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:17.861942 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7e387-b5b4-4826-9a9b-a5b43431a8bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:17.868991 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a7e387-b5b4-4826-9a9b-a5b43431a8bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:17.870442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a7e387-b5b4-4826-9a9b-a5b43431a8bd None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:17.884297 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application [None req-71a7e387-b5b4-4826-9a9b-a5b43431a8bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:47:17.884297 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:47:17.884297 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:47:17.884297 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:47:17.884297 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:47:17.884297 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:47:17.884297 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:47:17.884297 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:47:17.884297 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:47:17.884297 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:47:17.884297 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:47:17.884297 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:47:17.884297 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 01:47:17.884297 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 01:47:17.884297 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 01:47:17.885237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 01:47:17.885237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:47:17.885237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:47:17.885237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:47:17.885237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:47:17.885237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:47:17.885237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:47:17.885237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:47:17.885237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application with Lock( Jun 23 01:47:17.885237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:47:17.885237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:47:17.885237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:47:17.885237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:47:17.885237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:47:17.885237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:47:17.885237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:47:17.885237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:47:17.886178 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 01:47:17.886178 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 01:47:17.886178 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 01:47:17.886178 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 01:47:17.886178 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 01:47:17.886178 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 01:47:17.886178 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 01:47:17.886178 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: ERROR keystone.server.flask.application Jun 23 01:47:17.886178 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 647/1285] 10.222.0.26 () {56 vars in 1111 bytes} [Thu Jun 23 01:47:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 01:47:17.893962 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde314aa-333a-4ae1-b3d8-d6249567377e None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:17.948406 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:18.010317 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde314aa-333a-4ae1-b3d8-d6249567377e None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:18.046198 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21cfb2c-fe5a-4cf2-8b1d-83146c41e474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:18.046841 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21cfb2c-fe5a-4cf2-8b1d-83146c41e474 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:18.047153 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21cfb2c-fe5a-4cf2-8b1d-83146c41e474 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:18.047468 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21cfb2c-fe5a-4cf2-8b1d-83146c41e474 None admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:18.048283 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21cfb2c-fe5a-4cf2-8b1d-83146c41e474 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:47:18.048596 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21cfb2c-fe5a-4cf2-8b1d-83146c41e474 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:18.050046 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21cfb2c-fe5a-4cf2-8b1d-83146c41e474 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:18.064374 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 639/1286] 10.222.0.26 () {56 vars in 1148 bytes} [Thu Jun 23 01:47:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:47:18.070561 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a7e387-b5b4-4826-9a9b-a5b43431a8bd None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:18.174970 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a7e387-b5b4-4826-9a9b-a5b43431a8bd None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:18.212191 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a32c05-330f-4bf1-927d-02a07982bfc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:18.212833 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a32c05-330f-4bf1-927d-02a07982bfc2 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:18.213116 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a32c05-330f-4bf1-927d-02a07982bfc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:18.213391 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a32c05-330f-4bf1-927d-02a07982bfc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:18.214228 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a32c05-330f-4bf1-927d-02a07982bfc2 None admin] RBAC: Adding query filter params (role.id=7b20d65f49ee4a48894b6c060bb5d580, scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c, user.id=e5bf3baee1b843fa89b65e5485f6bebb) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:47:18.214547 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a32c05-330f-4bf1-927d-02a07982bfc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:18.216148 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a32c05-330f-4bf1-927d-02a07982bfc2 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:18.222941 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 648/1287] 10.222.0.26 () {56 vars in 1374 bytes} [Thu Jun 23 01:47:18 2022] GET /identity/v3/role_assignments?user.id=e5bf3baee1b843fa89b65e5485f6bebb&scope.project.id=a44a5e0e3e2d4b8e9b9cec7b5b5c725c&role.id=7b20d65f49ee4a48894b6c060bb5d580 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:47:20.235889 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12edf535-7920-4284-805b-57ca4642c054 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:20.236784 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12edf535-7920-4284-805b-57ca4642c054 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:20.236784 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12edf535-7920-4284-805b-57ca4642c054 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:20.238959 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 640/1288] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:47:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:47:20.245097 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fffef-db73-4062-be12-16ac86e97f51 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:20.245556 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fffef-db73-4062-be12-16ac86e97f51 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:20.245661 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fffef-db73-4062-be12-16ac86e97f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:20.270001 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-976fffef-db73-4062-be12-16ac86e97f51 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:47:20.271225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-976fffef-db73-4062-be12-16ac86e97f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:47:20.306088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:20.344727 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 649/1289] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:47:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:47:20.367720 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12edf535-7920-4284-805b-57ca4642c054 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:20.477139 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12edf535-7920-4284-805b-57ca4642c054 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:20.488162 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:20.516091 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7cb28-a83a-48d2-8962-c3438726f171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:20.516891 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7cb28-a83a-48d2-8962-c3438726f171 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:20.517174 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7cb28-a83a-48d2-8962-c3438726f171 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:20.517449 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7cb28-a83a-48d2-8962-c3438726f171 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:20.518103 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7cb28-a83a-48d2-8962-c3438726f171 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:20.519521 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7cb28-a83a-48d2-8962-c3438726f171 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:20.531777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-46b7cb28-a83a-48d2-8962-c3438726f171 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 01:47:20.531777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:47:20.531777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 01:47:20.531777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 01:47:20.531777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 01:47:20.531777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 01:47:20.531777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 01:47:20.531777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:47:20.531777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 01:47:20.531777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:47:20.531777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 01:47:20.531777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:47:20.531777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:47:20.531777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:47:20.531777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:47:20.531777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:47:20.531777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:47:20.532923 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:47:20.532923 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:47:20.532923 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:47:20.532923 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:47:20.532923 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:47:20.532923 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:47:20.532923 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 01:47:20.532923 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 01:47:20.532923 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 01:47:20.532923 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 01:47:20.532923 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:47:20.532923 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:47:20.532923 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:47:20.532923 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:47:20.532923 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:47:20.532923 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:47:20.533973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:47:20.533973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:47:20.533973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:47:20.533973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:47:20.533973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:47:20.533973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:47:20.533973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:47:20.533973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:47:20.533973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:47:20.533973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:47:20.533973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 01:47:20.533973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:47:20.533973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 01:47:20.533973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:47:20.533973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 641/1290] 10.222.0.26 () {56 vars in 1117 bytes} [Thu Jun 23 01:47:20 2022] GET /identity/v3/services/placement => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:47:20.539205 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976fffef-db73-4062-be12-16ac86e97f51 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:20.641391 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976fffef-db73-4062-be12-16ac86e97f51 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:20.676501 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0209777-3d4b-4518-877d-1ed3b2ed32b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:20.677148 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0209777-3d4b-4518-877d-1ed3b2ed32b7 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:20.677431 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0209777-3d4b-4518-877d-1ed3b2ed32b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:20.677704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0209777-3d4b-4518-877d-1ed3b2ed32b7 None admin] PATH_INFO: `/v3/services` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:20.678415 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0209777-3d4b-4518-877d-1ed3b2ed32b7 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:47:20.678722 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0209777-3d4b-4518-877d-1ed3b2ed32b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:20.680128 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0209777-3d4b-4518-877d-1ed3b2ed32b7 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:20.685145 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 650/1291] 10.222.0.26 () {56 vars in 1116 bytes} [Thu Jun 23 01:47:20 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:47:20.690904 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7cb28-a83a-48d2-8962-c3438726f171 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:20.800075 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7cb28-a83a-48d2-8962-c3438726f171 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:20.840693 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3af656-1644-496e-a48b-910b4e5ba904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:20.841423 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3af656-1644-496e-a48b-910b4e5ba904 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:20.841708 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3af656-1644-496e-a48b-910b4e5ba904 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:20.842013 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3af656-1644-496e-a48b-910b4e5ba904 None admin] PATH_INFO: `/v3/services` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:20.842736 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3af656-1644-496e-a48b-910b4e5ba904 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:47:20.843048 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3af656-1644-496e-a48b-910b4e5ba904 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:20.844477 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3af656-1644-496e-a48b-910b4e5ba904 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:20.849178 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 642/1292] 10.222.0.26 () {56 vars in 1116 bytes} [Thu Jun 23 01:47:20 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:47:22.844631 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d33ce0-d268-4cba-a18b-993f8540a84c None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:22.845521 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d33ce0-d268-4cba-a18b-993f8540a84c None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:22.845521 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d33ce0-d268-4cba-a18b-993f8540a84c None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:22.847768 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 651/1293] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:47:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:47:22.853920 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f0e1b-e2a1-4b2b-a175-7676102e4cbc None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:22.854651 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f0e1b-e2a1-4b2b-a175-7676102e4cbc None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:22.854806 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f0e1b-e2a1-4b2b-a175-7676102e4cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:22.880014 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-db5f0e1b-e2a1-4b2b-a175-7676102e4cbc None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:47:22.881248 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-db5f0e1b-e2a1-4b2b-a175-7676102e4cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:47:22.914949 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:22.954785 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 643/1294] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:47:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:47:22.977811 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d33ce0-d268-4cba-a18b-993f8540a84c None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:23.012591 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:23.091590 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d33ce0-d268-4cba-a18b-993f8540a84c None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:23.133908 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a171e3-e285-4805-a2a8-93cd540f5778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:23.134503 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a171e3-e285-4805-a2a8-93cd540f5778 None admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:23.134786 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a171e3-e285-4805-a2a8-93cd540f5778 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:23.135062 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a171e3-e285-4805-a2a8-93cd540f5778 None admin] PATH_INFO: `/v3/services` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:23.135834 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a171e3-e285-4805-a2a8-93cd540f5778 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:23.137237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a171e3-e285-4805-a2a8-93cd540f5778 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:23.152116 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 652/1295] 10.222.0.26 () {60 vars in 1141 bytes} [Thu Jun 23 01:47:22 2022] POST /identity/v3/services => generated 246 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:47:25.136327 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d147169-9f92-4b5f-87ac-8460ddb83b61 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:25.136892 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d147169-9f92-4b5f-87ac-8460ddb83b61 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:25.136892 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d147169-9f92-4b5f-87ac-8460ddb83b61 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:25.138266 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 644/1296] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:47:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:47:25.144543 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4b6d5-ef0e-4fa6-94b4-5e8f03cf6fc1 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:25.145224 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4b6d5-ef0e-4fa6-94b4-5e8f03cf6fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:25.145398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4b6d5-ef0e-4fa6-94b4-5e8f03cf6fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:25.165576 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-ece4b6d5-ef0e-4fa6-94b4-5e8f03cf6fc1 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:47:25.166762 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-ece4b6d5-ef0e-4fa6-94b4-5e8f03cf6fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:47:25.241776 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 653/1297] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:47:25 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:47:25.264985 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d147169-9f92-4b5f-87ac-8460ddb83b61 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:25.389146 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d147169-9f92-4b5f-87ac-8460ddb83b61 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:25.427731 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fecb37-afab-4fd8-b897-3f7cdc92a119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:25.428527 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fecb37-afab-4fd8-b897-3f7cdc92a119 None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:25.428803 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fecb37-afab-4fd8-b897-3f7cdc92a119 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:25.429075 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fecb37-afab-4fd8-b897-3f7cdc92a119 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:25.429729 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fecb37-afab-4fd8-b897-3f7cdc92a119 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:25.431107 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fecb37-afab-4fd8-b897-3f7cdc92a119 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:25.443240 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-42fecb37-afab-4fd8-b897-3f7cdc92a119 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 01:47:25.443240 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:47:25.443240 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 01:47:25.443240 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 01:47:25.443240 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 01:47:25.443240 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 01:47:25.443240 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 01:47:25.443240 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:47:25.443240 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 01:47:25.443240 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:47:25.443240 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 01:47:25.443240 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:47:25.443240 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:47:25.443240 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:47:25.443240 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:47:25.443240 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:47:25.443240 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:47:25.444385 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:47:25.444385 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:47:25.444385 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:47:25.444385 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:47:25.444385 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:47:25.444385 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:47:25.444385 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 01:47:25.444385 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 01:47:25.444385 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 01:47:25.444385 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 01:47:25.444385 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:47:25.444385 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:47:25.444385 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:47:25.444385 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:47:25.444385 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:47:25.444385 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:47:25.447782 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:47:25.447782 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:47:25.447782 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:47:25.447782 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:47:25.447782 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:47:25.447782 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:47:25.447782 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:47:25.447782 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:47:25.447782 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:47:25.447782 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:47:25.447782 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 01:47:25.447782 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:47:25.447782 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 01:47:25.447782 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:47:25.447782 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 645/1298] 10.222.0.26 () {56 vars in 1117 bytes} [Thu Jun 23 01:47:25 2022] GET /identity/v3/services/placement => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:47:25.451474 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece4b6d5-ef0e-4fa6-94b4-5e8f03cf6fc1 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:25.471281 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:25.555058 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece4b6d5-ef0e-4fa6-94b4-5e8f03cf6fc1 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:25.591777 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe26ee2a-9e4e-461f-827b-4b1fe9894368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:25.592426 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26ee2a-9e4e-461f-827b-4b1fe9894368 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:25.592686 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26ee2a-9e4e-461f-827b-4b1fe9894368 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:25.592989 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26ee2a-9e4e-461f-827b-4b1fe9894368 None admin] PATH_INFO: `/v3/services` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:25.593692 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe26ee2a-9e4e-461f-827b-4b1fe9894368 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:47:25.593997 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe26ee2a-9e4e-461f-827b-4b1fe9894368 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:25.595413 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe26ee2a-9e4e-461f-827b-4b1fe9894368 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:25.600729 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 654/1299] 10.222.0.26 () {56 vars in 1116 bytes} [Thu Jun 23 01:47:25 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:47:25.606855 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fecb37-afab-4fd8-b897-3f7cdc92a119 None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:25.639253 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:25.722179 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fecb37-afab-4fd8-b897-3f7cdc92a119 None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:25.757722 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251e249e-564f-43bb-a270-60ffc8daed3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:25.758396 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e249e-564f-43bb-a270-60ffc8daed3a None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:25.758672 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e249e-564f-43bb-a270-60ffc8daed3a None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:25.758997 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e249e-564f-43bb-a270-60ffc8daed3a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:25.759897 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251e249e-564f-43bb-a270-60ffc8daed3a None admin] RBAC: Adding query filter params (interface=public, service_id=c57aa87aa5704fb481738acef744587b, region_id=RegionOne) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:47:25.760206 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251e249e-564f-43bb-a270-60ffc8daed3a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:25.761589 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251e249e-564f-43bb-a270-60ffc8daed3a None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:25.766786 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 646/1300] 10.222.0.26 () {56 vars in 1251 bytes} [Thu Jun 23 01:47:25 2022] GET /identity/v3/endpoints?service_id=c57aa87aa5704fb481738acef744587b&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:47:25.773232 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe26ee2a-9e4e-461f-827b-4b1fe9894368 None admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:25.874594 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:25.896664 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe26ee2a-9e4e-461f-827b-4b1fe9894368 None admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:25.932827 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ef246-5895-48a7-8ac4-1ff679fe75aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:25.933478 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ef246-5895-48a7-8ac4-1ff679fe75aa None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:25.933753 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ef246-5895-48a7-8ac4-1ff679fe75aa None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:25.934025 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ef246-5895-48a7-8ac4-1ff679fe75aa None admin] PATH_INFO: `/v3/services` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:25.934717 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527ef246-5895-48a7-8ac4-1ff679fe75aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:25.936144 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527ef246-5895-48a7-8ac4-1ff679fe75aa None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:25.943035 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 655/1301] 10.222.0.26 () {56 vars in 1087 bytes} [Thu Jun 23 01:47:25 2022] GET /identity/v3/services => generated 2191 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:47:27.949647 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f2d9c-cb97-4ae8-9d40-ac2a2ef9a1dc None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:27.950232 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f2d9c-cb97-4ae8-9d40-ac2a2ef9a1dc None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:27.950232 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f2d9c-cb97-4ae8-9d40-ac2a2ef9a1dc None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:27.951637 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 647/1302] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:47:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:47:27.958142 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49ccff-88b6-41fc-bb41-b8a31c35c933 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:27.958830 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49ccff-88b6-41fc-bb41-b8a31c35c933 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:27.958964 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49ccff-88b6-41fc-bb41-b8a31c35c933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:27.985440 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-ff49ccff-88b6-41fc-bb41-b8a31c35c933 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:47:27.986633 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-ff49ccff-88b6-41fc-bb41-b8a31c35c933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:47:28.062591 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 656/1303] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:47:27 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 01:47:28.087726 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f2d9c-cb97-4ae8-9d40-ac2a2ef9a1dc None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:28.185862 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:28.193985 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f2d9c-cb97-4ae8-9d40-ac2a2ef9a1dc None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:28.229594 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae4398d-41bc-4714-b9dc-60424236139c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:28.230333 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4398d-41bc-4714-b9dc-60424236139c None admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:28.230617 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4398d-41bc-4714-b9dc-60424236139c None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:28.230893 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4398d-41bc-4714-b9dc-60424236139c None admin] PATH_INFO: `/v3/services/placement` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:28.231587 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae4398d-41bc-4714-b9dc-60424236139c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:28.232971 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae4398d-41bc-4714-b9dc-60424236139c None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:28.245257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application [None req-5ae4398d-41bc-4714-b9dc-60424236139c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 01:47:28.245257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:47:28.245257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 01:47:28.245257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 01:47:28.245257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 01:47:28.245257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 01:47:28.245257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 01:47:28.245257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:47:28.245257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 01:47:28.245257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:47:28.245257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 01:47:28.245257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:47:28.245257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 01:47:28.245257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 01:47:28.245257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 01:47:28.245257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 01:47:28.245257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 01:47:28.246399 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 01:47:28.246399 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 01:47:28.246399 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 01:47:28.246399 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 01:47:28.246399 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 01:47:28.246399 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 01:47:28.246399 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 01:47:28.246399 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 01:47:28.246399 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 01:47:28.246399 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 01:47:28.246399 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 01:47:28.246399 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 01:47:28.246399 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 01:47:28.246399 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 01:47:28.246399 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 01:47:28.246399 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 01:47:28.247436 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 01:47:28.247436 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application with Lock( Jun 23 01:47:28.247436 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 01:47:28.247436 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self._enter() Jun 23 01:47:28.247436 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 01:47:28.247436 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 01:47:28.247436 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 01:47:28.247436 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application return self.creator() Jun 23 01:47:28.247436 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 01:47:28.247436 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application created_value = creator( Jun 23 01:47:28.247436 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 01:47:28.247436 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 01:47:28.247436 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 01:47:28.247436 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: ERROR keystone.server.flask.application Jun 23 01:47:28.251424 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 648/1304] 10.222.0.26 () {56 vars in 1117 bytes} [Thu Jun 23 01:47:28 2022] GET /identity/v3/services/placement => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 01:47:28.253121 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff49ccff-88b6-41fc-bb41-b8a31c35c933 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:28.349013 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:28.373533 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff49ccff-88b6-41fc-bb41-b8a31c35c933 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:28.408843 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0779a2de-8a76-493a-aa04-d00558312729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:28.409500 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0779a2de-8a76-493a-aa04-d00558312729 None admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:28.409787 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0779a2de-8a76-493a-aa04-d00558312729 None admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:28.410075 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0779a2de-8a76-493a-aa04-d00558312729 None admin] PATH_INFO: `/v3/services` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:28.410796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0779a2de-8a76-493a-aa04-d00558312729 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:47:28.411107 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0779a2de-8a76-493a-aa04-d00558312729 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:28.412527 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0779a2de-8a76-493a-aa04-d00558312729 None admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:28.417794 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 657/1305] 10.222.0.26 () {56 vars in 1116 bytes} [Thu Jun 23 01:47:28 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:47:28.423907 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae4398d-41bc-4714-b9dc-60424236139c None admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:47:28.533831 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae4398d-41bc-4714-b9dc-60424236139c None admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:47:28.571062 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ae521d-2c9b-4f86-8d76-36d881cbd442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:47:28.571745 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae521d-2c9b-4f86-8d76-36d881cbd442 None admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:47:28.572023 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae521d-2c9b-4f86-8d76-36d881cbd442 None admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:47:28.572300 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae521d-2c9b-4f86-8d76-36d881cbd442 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:47:28.573064 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ae521d-2c9b-4f86-8d76-36d881cbd442 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:47:28.574444 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ae521d-2c9b-4f86-8d76-36d881cbd442 None admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:47:28.597006 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 649/1306] 10.222.0.26 () {60 vars in 1144 bytes} [Thu Jun 23 01:47:28 2022] POST /identity/v3/endpoints => generated 331 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:47:38.414737 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=30 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:47:38.592242 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:48:31.563610 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472d678-862c-489f-be89-4457cfbc431f None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:48:31.564155 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472d678-862c-489f-be89-4457cfbc431f None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:48:31.564155 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472d678-862c-489f-be89-4457cfbc431f None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:48:31.565745 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 658/1307] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:48:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:48:31.570661 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c2064-337b-4227-aa10-dbcbb97a9742 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:48:31.571133 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c2064-337b-4227-aa10-dbcbb97a9742 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:48:31.571256 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c2064-337b-4227-aa10-dbcbb97a9742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:48:31.659169 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-e35c2064-337b-4227-aa10-dbcbb97a9742 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:48:31.660808 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-e35c2064-337b-4227-aa10-dbcbb97a9742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:48:31.803315 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 650/1308] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:48:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:48:31.852202 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c4315-2592-4d2f-b536-ac09945971f2 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:48:31.852494 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c4315-2592-4d2f-b536-ac09945971f2 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:48:31.852800 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c4315-2592-4d2f-b536-ac09945971f2 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:48:31.854245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 659/1309] 10.222.0.26 () {54 vars in 964 bytes} [Thu Jun 23 01:48:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:48:31.860545 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d20ea-b466-4194-93ad-ef15302505b1 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:48:31.860843 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d20ea-b466-4194-93ad-ef15302505b1 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:48:31.861131 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d20ea-b466-4194-93ad-ef15302505b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:48:32.017615 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-808d20ea-b466-4194-93ad-ef15302505b1 None None] MFA Rules not processed for user `948e4905de44473bb7a45e1b521cf095`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:48:32.018843 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-808d20ea-b466-4194-93ad-ef15302505b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:48:32.044518 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:48:32.105914 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 651/1310] 10.222.0.26 () {58 vars in 1061 bytes} [Thu Jun 23 01:48:31 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 01:48:32.115000 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c4315-2592-4d2f-b536-ac09945971f2 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:48:32.116830 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-0c4c4315-2592-4d2f-b536-ac09945971f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:48:32.245556 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c4315-2592-4d2f-b536-ac09945971f2 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:48:32.339626 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b49a0f9-dcab-4acc-a28f-298e99aefc42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:48:32.340311 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b49a0f9-dcab-4acc-a28f-298e99aefc42 service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:48:32.340601 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b49a0f9-dcab-4acc-a28f-298e99aefc42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:48:32.340844 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b49a0f9-dcab-4acc-a28f-298e99aefc42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:48:32.342553 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-9b49a0f9-dcab-4acc-a28f-298e99aefc42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:48:32.404165 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:48:32.471877 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b49a0f9-dcab-4acc-a28f-298e99aefc42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:48:32.473375 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b49a0f9-dcab-4acc-a28f-298e99aefc42 service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:48:32.494847 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 660/1311] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:48:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:48:34.813700 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5060fa3-9122-4427-9a94-fa4da44da7f3 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:48:34.814222 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5060fa3-9122-4427-9a94-fa4da44da7f3 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:48:34.814301 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5060fa3-9122-4427-9a94-fa4da44da7f3 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:48:34.816082 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 652/1312] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:48:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:48:34.822128 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c851a7-7533-472c-afd6-b320ba060c68 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:48:34.822671 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c851a7-7533-472c-afd6-b320ba060c68 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:48:34.822671 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c851a7-7533-472c-afd6-b320ba060c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:48:34.902942 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-00c851a7-7533-472c-afd6-b320ba060c68 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:48:34.904263 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-00c851a7-7533-472c-afd6-b320ba060c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:48:35.005240 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 661/1313] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:48:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:48:35.043242 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5060fa3-9122-4427-9a94-fa4da44da7f3 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:48:35.044928 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-d5060fa3-9122-4427-9a94-fa4da44da7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:48:35.158923 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5060fa3-9122-4427-9a94-fa4da44da7f3 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:48:35.233631 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:48:35.236727 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b979fc-01f3-4049-80db-6df85a01a2b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:48:35.237382 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b979fc-01f3-4049-80db-6df85a01a2b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:48:35.237662 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b979fc-01f3-4049-80db-6df85a01a2b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:48:35.237940 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b979fc-01f3-4049-80db-6df85a01a2b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:48:35.240107 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-b1b979fc-01f3-4049-80db-6df85a01a2b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:48:35.356292 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b979fc-01f3-4049-80db-6df85a01a2b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:48:35.357663 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b979fc-01f3-4049-80db-6df85a01a2b1 service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:48:35.376207 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 653/1314] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:48:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:48:37.692682 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd286e5b-bf56-404d-baa2-f3a7d4f9931d None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:48:37.693240 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd286e5b-bf56-404d-baa2-f3a7d4f9931d None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:48:37.693240 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd286e5b-bf56-404d-baa2-f3a7d4f9931d None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:48:37.694620 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 662/1315] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:48:37.700051 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e19e15c-0545-4de8-aa1e-67704b716a47 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:48:37.700462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e19e15c-0545-4de8-aa1e-67704b716a47 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:48:37.700553 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e19e15c-0545-4de8-aa1e-67704b716a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:48:37.773760 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-1e19e15c-0545-4de8-aa1e-67704b716a47 None None] MFA Rules not processed for user `dce468546a1841c0bc2f6bda45393ccd`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:48:37.774986 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-1e19e15c-0545-4de8-aa1e-67704b716a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:48:37.829513 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:48:37.879243 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 654/1316] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:48:37 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:48:37.918650 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd286e5b-bf56-404d-baa2-f3a7d4f9931d None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:48:37.920494 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-cd286e5b-bf56-404d-baa2-f3a7d4f9931d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:48:37.936606 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:48:38.051323 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd286e5b-bf56-404d-baa2-f3a7d4f9931d None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:48:38.137247 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f2af8-9686-4cca-92e3-1ef5fa4b0bc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:48:38.137898 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f2af8-9686-4cca-92e3-1ef5fa4b0bc1 service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:48:38.138178 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f2af8-9686-4cca-92e3-1ef5fa4b0bc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:48:38.138455 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f2af8-9686-4cca-92e3-1ef5fa4b0bc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:48:38.140207 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-6d8f2af8-9686-4cca-92e3-1ef5fa4b0bc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:48:38.207771 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:48:38.255615 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8f2af8-9686-4cca-92e3-1ef5fa4b0bc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:48:38.257062 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8f2af8-9686-4cca-92e3-1ef5fa4b0bc1 service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:48:38.275945 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 663/1317] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:48:37 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:48:40.802014 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5725a-38ce-4634-80d3-b3b6c4ee763d None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:48:40.802883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5725a-38ce-4634-80d3-b3b6c4ee763d None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:48:40.802883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5725a-38ce-4634-80d3-b3b6c4ee763d None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:48:40.805201 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 655/1318] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:48:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:48:40.813302 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b9a6d-08df-48b0-98fb-54fb460816fe None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:48:40.813780 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b9a6d-08df-48b0-98fb-54fb460816fe None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:48:40.813876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b9a6d-08df-48b0-98fb-54fb460816fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:48:40.884101 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-444b9a6d-08df-48b0-98fb-54fb460816fe None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:48:40.885303 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-444b9a6d-08df-48b0-98fb-54fb460816fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:48:40.986398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 664/1319] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:48:40 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 01:48:41.042747 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce7c1ad-42a1-4dad-ace3-cea8a58e652f None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:48:41.043229 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce7c1ad-42a1-4dad-ace3-cea8a58e652f None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:48:41.043527 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce7c1ad-42a1-4dad-ace3-cea8a58e652f None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:48:41.044921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 656/1320] 10.222.0.26 () {54 vars in 964 bytes} [Thu Jun 23 01:48:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 01:48:41.051390 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c0d06-bf8e-4f62-9c25-845c0cfce370 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:48:41.051779 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c0d06-bf8e-4f62-9c25-845c0cfce370 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:48:41.052528 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c0d06-bf8e-4f62-9c25-845c0cfce370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:48:41.077349 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:48:41.149247 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-092c0d06-bf8e-4f62-9c25-845c0cfce370 None None] MFA Rules not processed for user `948e4905de44473bb7a45e1b521cf095`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:48:41.150469 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-092c0d06-bf8e-4f62-9c25-845c0cfce370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:48:41.238762 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 665/1321] 10.222.0.26 () {58 vars in 1061 bytes} [Thu Jun 23 01:48:41 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:48:41.248781 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce7c1ad-42a1-4dad-ace3-cea8a58e652f None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:48:41.251179 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-5ce7c1ad-42a1-4dad-ace3-cea8a58e652f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:48:41.367280 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce7c1ad-42a1-4dad-ace3-cea8a58e652f None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:48:41.448516 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2e4081-57b1-4ac6-956e-ec9448e24e9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:48:41.449165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2e4081-57b1-4ac6-956e-ec9448e24e9b service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:48:41.449440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2e4081-57b1-4ac6-956e-ec9448e24e9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:48:41.449717 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2e4081-57b1-4ac6-956e-ec9448e24e9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:48:41.451417 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-6c2e4081-57b1-4ac6-956e-ec9448e24e9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:48:41.459945 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:48:41.568838 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2e4081-57b1-4ac6-956e-ec9448e24e9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:48:41.570271 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2e4081-57b1-4ac6-956e-ec9448e24e9b service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:48:41.588661 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 657/1322] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:48:41 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:48:44.748127 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106b6d9-fcb2-464f-94eb-891add567629 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:48:44.748717 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106b6d9-fcb2-464f-94eb-891add567629 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:48:44.748717 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106b6d9-fcb2-464f-94eb-891add567629 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:48:44.750115 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 666/1323] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:48:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:48:44.755741 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2515bf40-5ac5-4dc7-a598-9e9579ab1ea3 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:48:44.756200 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2515bf40-5ac5-4dc7-a598-9e9579ab1ea3 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:48:44.756278 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2515bf40-5ac5-4dc7-a598-9e9579ab1ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:48:44.821876 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-2515bf40-5ac5-4dc7-a598-9e9579ab1ea3 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:48:44.823079 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-2515bf40-5ac5-4dc7-a598-9e9579ab1ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:48:44.885513 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:48:44.930924 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 658/1324] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:48:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 01:48:44.967566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d106b6d9-fcb2-464f-94eb-891add567629 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:48:44.969300 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-d106b6d9-fcb2-464f-94eb-891add567629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:48:45.090248 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d106b6d9-fcb2-464f-94eb-891add567629 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:48:45.102720 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:48:45.166727 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480e3f41-795c-4d8d-841a-c8522c710bae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:48:45.167405 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e3f41-795c-4d8d-841a-c8522c710bae service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:48:45.167680 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e3f41-795c-4d8d-841a-c8522c710bae service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:48:45.167952 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e3f41-795c-4d8d-841a-c8522c710bae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:48:45.169593 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-480e3f41-795c-4d8d-841a-c8522c710bae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:48:45.285638 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480e3f41-795c-4d8d-841a-c8522c710bae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:48:45.286987 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480e3f41-795c-4d8d-841a-c8522c710bae service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:48:45.307375 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 667/1325] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:48:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 01:48:47.379360 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b4e3a7-8eb4-4c33-8255-a07a406c91c4 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:48:47.379889 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b4e3a7-8eb4-4c33-8255-a07a406c91c4 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:48:47.379889 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b4e3a7-8eb4-4c33-8255-a07a406c91c4 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:48:47.381211 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 659/1326] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:48:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:48:47.386488 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c85ac82-81ca-4624-90fe-51811fd44d0e None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:48:47.386912 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c85ac82-81ca-4624-90fe-51811fd44d0e None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:48:47.386999 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c85ac82-81ca-4624-90fe-51811fd44d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:48:47.434667 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:48:47.453471 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-0c85ac82-81ca-4624-90fe-51811fd44d0e None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:48:47.454694 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-0c85ac82-81ca-4624-90fe-51811fd44d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:48:47.554735 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 668/1327] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:48:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:48:47.590950 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b4e3a7-8eb4-4c33-8255-a07a406c91c4 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:48:47.592736 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-b3b4e3a7-8eb4-4c33-8255-a07a406c91c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:48:47.700289 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b4e3a7-8eb4-4c33-8255-a07a406c91c4 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:48:47.782034 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b20994-37cd-4a6c-ac1d-af71a45d61c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:48:47.782699 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b20994-37cd-4a6c-ac1d-af71a45d61c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:48:47.782976 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b20994-37cd-4a6c-ac1d-af71a45d61c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:48:47.783272 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b20994-37cd-4a6c-ac1d-af71a45d61c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:48:47.784923 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-f6b20994-37cd-4a6c-ac1d-af71a45d61c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:48:47.803241 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:48:47.906921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b20994-37cd-4a6c-ac1d-af71a45d61c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:48:47.908331 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b20994-37cd-4a6c-ac1d-af71a45d61c4 service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:48:47.928209 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 660/1328] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:48:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:48:49.983075 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c6086e-69fb-4f7f-aef4-6579d2630f71 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:48:49.983731 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c6086e-69fb-4f7f-aef4-6579d2630f71 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:48:49.983731 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c6086e-69fb-4f7f-aef4-6579d2630f71 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:48:49.985019 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 669/1329] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:48:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:48:49.990586 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca175f16-e7ae-4caf-8582-a63b44d331a0 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:48:49.991065 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca175f16-e7ae-4caf-8582-a63b44d331a0 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:48:49.991169 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca175f16-e7ae-4caf-8582-a63b44d331a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:48:50.058228 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-ca175f16-e7ae-4caf-8582-a63b44d331a0 None None] MFA Rules not processed for user `dce468546a1841c0bc2f6bda45393ccd`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:48:50.059513 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-ca175f16-e7ae-4caf-8582-a63b44d331a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:48:50.136698 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:48:50.170495 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 661/1330] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:48:49 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:48:50.208502 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c6086e-69fb-4f7f-aef4-6579d2630f71 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:48:50.210040 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-05c6086e-69fb-4f7f-aef4-6579d2630f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:48:50.322689 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c6086e-69fb-4f7f-aef4-6579d2630f71 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:48:50.342872 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:48:50.397410 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e259ff-1fbc-44c5-965e-6fbf653d72c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:48:50.398051 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e259ff-1fbc-44c5-965e-6fbf653d72c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:48:50.398334 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e259ff-1fbc-44c5-965e-6fbf653d72c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:48:50.398601 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e259ff-1fbc-44c5-965e-6fbf653d72c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:48:50.400289 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-51e259ff-1fbc-44c5-965e-6fbf653d72c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:48:50.529534 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e259ff-1fbc-44c5-965e-6fbf653d72c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:48:50.531134 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e259ff-1fbc-44c5-965e-6fbf653d72c7 service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:48:50.550633 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 670/1331] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:48:50 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:48:52.816169 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b66fa-132e-4fed-8fbd-43bcb2382174 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:48:52.817060 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b66fa-132e-4fed-8fbd-43bcb2382174 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:48:52.817060 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b66fa-132e-4fed-8fbd-43bcb2382174 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:48:52.819178 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 662/1332] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:48:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:48:52.827934 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b005c508-0611-4c67-81c8-29f12cf45ce6 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:48:52.828370 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b005c508-0611-4c67-81c8-29f12cf45ce6 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:48:52.828506 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b005c508-0611-4c67-81c8-29f12cf45ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:48:52.899075 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:48:52.901898 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-b005c508-0611-4c67-81c8-29f12cf45ce6 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:48:52.903663 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-b005c508-0611-4c67-81c8-29f12cf45ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:48:53.004986 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 671/1333] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:48:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:48:53.044090 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b66fa-132e-4fed-8fbd-43bcb2382174 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:48:53.046548 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-c00b66fa-132e-4fed-8fbd-43bcb2382174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:48:53.160506 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b66fa-132e-4fed-8fbd-43bcb2382174 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:48:53.237227 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276dec43-1582-4bc9-8270-311e24fc8289 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:48:53.237877 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276dec43-1582-4bc9-8270-311e24fc8289 service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:48:53.238153 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276dec43-1582-4bc9-8270-311e24fc8289 service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:48:53.238425 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276dec43-1582-4bc9-8270-311e24fc8289 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:48:53.240104 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-276dec43-1582-4bc9-8270-311e24fc8289 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:48:53.267185 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:48:53.356038 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276dec43-1582-4bc9-8270-311e24fc8289 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:48:53.357492 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276dec43-1582-4bc9-8270-311e24fc8289 service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:48:53.376824 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 663/1334] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:48:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:48:55.412211 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f741a571-5de4-4bc5-a2da-8640e471beeb None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:48:55.413054 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f741a571-5de4-4bc5-a2da-8640e471beeb None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:48:55.413054 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f741a571-5de4-4bc5-a2da-8640e471beeb None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:48:55.415249 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 672/1335] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:48:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:48:55.423041 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f036e65-cb71-441d-bce6-a75192dc94f6 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:48:55.423508 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f036e65-cb71-441d-bce6-a75192dc94f6 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:48:55.423709 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f036e65-cb71-441d-bce6-a75192dc94f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:48:55.500202 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-3f036e65-cb71-441d-bce6-a75192dc94f6 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:48:55.501443 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-3f036e65-cb71-441d-bce6-a75192dc94f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:48:55.578282 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:48:55.611997 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 664/1336] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:48:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:48:55.656108 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f741a571-5de4-4bc5-a2da-8640e471beeb None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:48:55.658497 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-f741a571-5de4-4bc5-a2da-8640e471beeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:48:55.770610 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f741a571-5de4-4bc5-a2da-8640e471beeb None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:48:55.801381 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:48:55.849672 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a44bd-919d-4611-9590-2be15f6c47a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:48:55.850315 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a44bd-919d-4611-9590-2be15f6c47a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:48:55.850589 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a44bd-919d-4611-9590-2be15f6c47a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:48:55.850854 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a44bd-919d-4611-9590-2be15f6c47a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:48:55.852542 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-903a44bd-919d-4611-9590-2be15f6c47a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:48:55.964936 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903a44bd-919d-4611-9590-2be15f6c47a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:48:55.966391 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903a44bd-919d-4611-9590-2be15f6c47a6 service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:48:55.984711 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 673/1337] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:48:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:48:58.404880 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8b11d-3eaa-4b03-b3c1-cedb342692bf None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:48:58.405401 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8b11d-3eaa-4b03-b3c1-cedb342692bf None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:48:58.405508 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8b11d-3eaa-4b03-b3c1-cedb342692bf None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:48:58.406870 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 665/1338] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:48:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:48:58.412472 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20b8b8-6a0f-47e8-8786-d9b559584d9f None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:48:58.412852 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20b8b8-6a0f-47e8-8786-d9b559584d9f None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:48:58.413028 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20b8b8-6a0f-47e8-8786-d9b559584d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:48:58.491878 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-4d20b8b8-6a0f-47e8-8786-d9b559584d9f None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:48:58.493113 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-4d20b8b8-6a0f-47e8-8786-d9b559584d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:48:58.501851 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:48:58.596962 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 674/1339] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:48:58 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:48:58.633388 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8b11d-3eaa-4b03-b3c1-cedb342692bf None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:48:58.634941 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-62c8b11d-3eaa-4b03-b3c1-cedb342692bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:48:58.759689 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8b11d-3eaa-4b03-b3c1-cedb342692bf None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:48:58.850684 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b2a827-5e49-460c-baaa-ea255fcf6774 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:48:58.851263 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2a827-5e49-460c-baaa-ea255fcf6774 service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:48:58.851579 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2a827-5e49-460c-baaa-ea255fcf6774 service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:48:58.851855 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2a827-5e49-460c-baaa-ea255fcf6774 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:48:58.853515 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-e8b2a827-5e49-460c-baaa-ea255fcf6774 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:48:58.890449 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:48:58.982324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b2a827-5e49-460c-baaa-ea255fcf6774 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:48:58.983871 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b2a827-5e49-460c-baaa-ea255fcf6774 service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:48:59.005682 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 666/1340] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:48:58 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 01:49:02.931104 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3035225-4d38-4c44-aee3-4511916b9907 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:02.931725 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3035225-4d38-4c44-aee3-4511916b9907 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:02.931725 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3035225-4d38-4c44-aee3-4511916b9907 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:02.933061 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 675/1341] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:49:02.938344 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c786f2-6f6d-440f-b0e0-f43955420b14 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:02.938789 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c786f2-6f6d-440f-b0e0-f43955420b14 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:02.938887 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c786f2-6f6d-440f-b0e0-f43955420b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:03.012571 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-91c786f2-6f6d-440f-b0e0-f43955420b14 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:49:03.013808 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-91c786f2-6f6d-440f-b0e0-f43955420b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:49:03.111999 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:49:03.126220 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 667/1342] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:49:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:49:03.164917 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3035225-4d38-4c44-aee3-4511916b9907 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:49:03.167465 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-f3035225-4d38-4c44-aee3-4511916b9907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:49:03.270884 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3035225-4d38-4c44-aee3-4511916b9907 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:49:03.274487 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c99abda-0b35-46af-91ec-98467e484c57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:49:03.275337 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c99abda-0b35-46af-91ec-98467e484c57 service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:03.275616 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c99abda-0b35-46af-91ec-98467e484c57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:03.275886 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c99abda-0b35-46af-91ec-98467e484c57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:03.285206 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c99abda-0b35-46af-91ec-98467e484c57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:49:03.286648 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c99abda-0b35-46af-91ec-98467e484c57 service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:49:03.299517 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 676/1343] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:49:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:49:05.872061 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d53883-c7c2-4cd3-8067-e1f0c72f1678 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:05.872651 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d53883-c7c2-4cd3-8067-e1f0c72f1678 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:05.872651 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d53883-c7c2-4cd3-8067-e1f0c72f1678 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:05.874046 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 668/1344] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:49:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:49:05.879505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09ee05-9dae-4b37-80c9-d34a0375c9eb None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:05.879925 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09ee05-9dae-4b37-80c9-d34a0375c9eb None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:05.880037 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09ee05-9dae-4b37-80c9-d34a0375c9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:05.909509 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-7c09ee05-9dae-4b37-80c9-d34a0375c9eb None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:49:05.910695 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-7c09ee05-9dae-4b37-80c9-d34a0375c9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:49:05.941664 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:49:05.991413 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 677/1345] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:49:05 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:49:06.028525 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d53883-c7c2-4cd3-8067-e1f0c72f1678 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:49:06.041649 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d53883-c7c2-4cd3-8067-e1f0c72f1678 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:49:06.046122 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcc340a-b734-4815-9003-bd3357108f21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:49:06.046968 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcc340a-b734-4815-9003-bd3357108f21 service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:06.047352 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcc340a-b734-4815-9003-bd3357108f21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:06.047688 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcc340a-b734-4815-9003-bd3357108f21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:06.058877 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcc340a-b734-4815-9003-bd3357108f21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:49:06.060484 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcc340a-b734-4815-9003-bd3357108f21 service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:49:06.071913 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 669/1346] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:49:06 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:49:10.100580 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d33488-1ef2-49f5-bdba-91d568bcb921 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:10.101115 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d33488-1ef2-49f5-bdba-91d568bcb921 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:10.101115 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d33488-1ef2-49f5-bdba-91d568bcb921 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:10.102571 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 678/1347] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:49:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:49:10.107921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf2924-f1d5-4a8c-a917-f8838fdc465b None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:10.108407 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf2924-f1d5-4a8c-a917-f8838fdc465b None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:10.108511 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf2924-f1d5-4a8c-a917-f8838fdc465b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:10.126689 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-1fcf2924-f1d5-4a8c-a917-f8838fdc465b None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:49:10.127971 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-1fcf2924-f1d5-4a8c-a917-f8838fdc465b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:49:10.137480 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 670/1348] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:49:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:49:10.174988 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d33488-1ef2-49f5-bdba-91d568bcb921 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:49:10.186764 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d33488-1ef2-49f5-bdba-91d568bcb921 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:49:10.190339 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2783f60-a1e4-4071-a75d-fd96bf1b29f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:49:10.190972 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2783f60-a1e4-4071-a75d-fd96bf1b29f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:10.191271 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2783f60-a1e4-4071-a75d-fd96bf1b29f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:10.191542 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2783f60-a1e4-4071-a75d-fd96bf1b29f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:10.200879 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2783f60-a1e4-4071-a75d-fd96bf1b29f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:49:10.202281 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2783f60-a1e4-4071-a75d-fd96bf1b29f8 service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:49:10.213111 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 679/1349] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:49:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:49:15.105763 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667396d-d052-4a80-bbc2-bab0b6aa06f8 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:15.106260 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667396d-d052-4a80-bbc2-bab0b6aa06f8 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:15.106260 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667396d-d052-4a80-bbc2-bab0b6aa06f8 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:15.111846 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 671/1350] 10.222.0.26 () {54 vars in 921 bytes} [Thu Jun 23 01:49:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:49:15.118257 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc439d-e64d-4796-b86e-08cc70d93373 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:15.118732 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc439d-e64d-4796-b86e-08cc70d93373 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:15.118997 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc439d-e64d-4796-b86e-08cc70d93373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:15.245545 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-0fdc439d-e64d-4796-b86e-08cc70d93373 None None] MFA Rules not processed for user `e5bf3baee1b843fa89b65e5485f6bebb`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:49:15.247179 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-0fdc439d-e64d-4796-b86e-08cc70d93373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:49:15.348090 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 680/1351] 10.222.0.26 () {58 vars in 1018 bytes} [Thu Jun 23 01:49:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:49:19.505707 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e668a-fc80-4bbc-b7c2-4bc111e9e26d None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:19.506253 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e668a-fc80-4bbc-b7c2-4bc111e9e26d None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:19.506253 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e668a-fc80-4bbc-b7c2-4bc111e9e26d None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:19.507959 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 672/1352] 10.222.0.26 () {54 vars in 921 bytes} [Thu Jun 23 01:49:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:49:19.513250 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe68376-6f3d-451c-8397-bab77a8b2320 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:19.513697 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe68376-6f3d-451c-8397-bab77a8b2320 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:19.513931 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe68376-6f3d-451c-8397-bab77a8b2320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:19.533769 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-6fe68376-6f3d-451c-8397-bab77a8b2320 None None] MFA Rules not processed for user `e5bf3baee1b843fa89b65e5485f6bebb`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:49:19.535099 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-6fe68376-6f3d-451c-8397-bab77a8b2320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:49:19.545180 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 681/1353] 10.222.0.26 () {58 vars in 1018 bytes} [Thu Jun 23 01:49:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:49:20.121234 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=13 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:49:21.106781 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4a0b2-5900-48c5-888b-2e70e84c9447 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:21.107510 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4a0b2-5900-48c5-888b-2e70e84c9447 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:21.107510 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4a0b2-5900-48c5-888b-2e70e84c9447 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:21.108805 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 673/1354] 10.222.0.26 () {54 vars in 921 bytes} [Thu Jun 23 01:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:49:21.113914 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b7485-7ad6-40d9-80c9-25742494afc1 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:21.114392 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b7485-7ad6-40d9-80c9-25742494afc1 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:21.114624 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b7485-7ad6-40d9-80c9-25742494afc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:21.133698 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-d63b7485-7ad6-40d9-80c9-25742494afc1 None None] MFA Rules not processed for user `e5bf3baee1b843fa89b65e5485f6bebb`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:49:21.135025 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-d63b7485-7ad6-40d9-80c9-25742494afc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:49:21.144724 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 682/1355] 10.222.0.26 () {58 vars in 1018 bytes} [Thu Jun 23 01:49:21 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:49:24.710880 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4b381-d34b-419b-8efa-a39a1e4b7586 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:24.711430 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4b381-d34b-419b-8efa-a39a1e4b7586 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:24.711430 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4b381-d34b-419b-8efa-a39a1e4b7586 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:24.713093 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 674/1356] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:49:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:49:24.719410 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a214827-b573-45a7-832d-937a9b767ee6 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:24.720234 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a214827-b573-45a7-832d-937a9b767ee6 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:24.720717 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a214827-b573-45a7-832d-937a9b767ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:24.746150 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-4a214827-b573-45a7-832d-937a9b767ee6 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:49:24.747760 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-4a214827-b573-45a7-832d-937a9b767ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:49:24.756826 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 683/1357] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:49:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:49:24.815926 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd1de0-b8dc-4890-a935-0b50a47c6775 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:24.816434 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd1de0-b8dc-4890-a935-0b50a47c6775 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:24.816927 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd1de0-b8dc-4890-a935-0b50a47c6775 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:24.818534 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 675/1358] 10.222.0.26 () {54 vars in 961 bytes} [Thu Jun 23 01:49:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:49:24.835974 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07e0a2-3e85-4a2c-b209-5869a44eecb2 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:24.836170 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07e0a2-3e85-4a2c-b209-5869a44eecb2 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:24.836434 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07e0a2-3e85-4a2c-b209-5869a44eecb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:24.881474 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:49:24.891581 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-8b07e0a2-3e85-4a2c-b209-5869a44eecb2 None None] MFA Rules not processed for user `ae4e83df33b64a3ebf28f54e2d94fe63`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:49:24.893140 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-8b07e0a2-3e85-4a2c-b209-5869a44eecb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:49:24.984112 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 684/1359] 10.222.0.26 () {58 vars in 1058 bytes} [Thu Jun 23 01:49:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:49:24.992292 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecd1de0-b8dc-4890-a935-0b50a47c6775 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:49:25.007660 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecd1de0-b8dc-4890-a935-0b50a47c6775 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:49:25.013073 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c808a41-7206-48fe-8137-eda0ca5f6773 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4e83df33b64a3ebf28f54e2d94fe63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:49:25.013836 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c808a41-7206-48fe-8137-eda0ca5f6773 service nova] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:25.014390 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c808a41-7206-48fe-8137-eda0ca5f6773 service nova] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:25.014730 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c808a41-7206-48fe-8137-eda0ca5f6773 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:25.023427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c808a41-7206-48fe-8137-eda0ca5f6773 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:49:25.025112 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c808a41-7206-48fe-8137-eda0ca5f6773 service nova] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:49:25.035157 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 676/1360] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:49:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:49:27.269619 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc2558-02a6-411f-98d8-abfe5e91fb63 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:27.270170 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc2558-02a6-411f-98d8-abfe5e91fb63 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:27.270170 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc2558-02a6-411f-98d8-abfe5e91fb63 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:27.274877 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 685/1361] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:49:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:49:27.277130 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913e726-a55b-4fa3-ae31-4773cbe390f3 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:27.277597 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913e726-a55b-4fa3-ae31-4773cbe390f3 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:27.277696 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913e726-a55b-4fa3-ae31-4773cbe390f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:27.296925 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-a913e726-a55b-4fa3-ae31-4773cbe390f3 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:49:27.298285 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-a913e726-a55b-4fa3-ae31-4773cbe390f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:49:27.311293 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 677/1362] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:49:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:49:27.352063 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cc2558-02a6-411f-98d8-abfe5e91fb63 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:49:27.366351 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cc2558-02a6-411f-98d8-abfe5e91fb63 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:49:27.371681 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29710946-00be-4453-bda4-584470a24c4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4e83df33b64a3ebf28f54e2d94fe63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:49:27.372308 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29710946-00be-4453-bda4-584470a24c4c service nova] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:27.372588 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29710946-00be-4453-bda4-584470a24c4c service nova] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:27.372865 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29710946-00be-4453-bda4-584470a24c4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:27.381467 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29710946-00be-4453-bda4-584470a24c4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:49:27.383102 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29710946-00be-4453-bda4-584470a24c4c service nova] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:49:27.393016 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 686/1363] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:49:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:49:29.737243 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99806eff-663d-47cd-bc3e-fa8596c1e6c0 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:29.738083 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99806eff-663d-47cd-bc3e-fa8596c1e6c0 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:29.738546 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99806eff-663d-47cd-bc3e-fa8596c1e6c0 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:29.741621 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 678/1364] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:49:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:49:29.746430 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c9608-be9b-4b97-b463-59132795b509 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:29.747277 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c9608-be9b-4b97-b463-59132795b509 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:29.747706 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c9608-be9b-4b97-b463-59132795b509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:29.766307 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-3b8c9608-be9b-4b97-b463-59132795b509 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:49:29.767587 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-3b8c9608-be9b-4b97-b463-59132795b509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:49:29.775783 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 687/1365] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:49:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:49:29.822486 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a566d7-81d9-4077-a460-06bcb1b306cd None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:29.822864 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a566d7-81d9-4077-a460-06bcb1b306cd None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:29.823245 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a566d7-81d9-4077-a460-06bcb1b306cd None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:29.824672 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 679/1366] 10.222.0.26 () {54 vars in 961 bytes} [Thu Jun 23 01:49:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:49:29.830280 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292a5e1-7807-4b75-9023-aec02605e0e5 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:29.830564 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292a5e1-7807-4b75-9023-aec02605e0e5 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:29.830844 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292a5e1-7807-4b75-9023-aec02605e0e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:29.850247 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-2292a5e1-7807-4b75-9023-aec02605e0e5 None None] MFA Rules not processed for user `ae4e83df33b64a3ebf28f54e2d94fe63`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:49:29.851460 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-2292a5e1-7807-4b75-9023-aec02605e0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:49:29.863157 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 688/1367] 10.222.0.26 () {58 vars in 1058 bytes} [Thu Jun 23 01:49:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:49:29.871439 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a566d7-81d9-4077-a460-06bcb1b306cd None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:49:29.885877 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a566d7-81d9-4077-a460-06bcb1b306cd None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:49:29.891598 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7548ab46-7548-4a0f-968a-25f7c64cbe5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4e83df33b64a3ebf28f54e2d94fe63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:49:29.892229 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7548ab46-7548-4a0f-968a-25f7c64cbe5c service nova] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:29.892509 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7548ab46-7548-4a0f-968a-25f7c64cbe5c service nova] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:29.892783 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7548ab46-7548-4a0f-968a-25f7c64cbe5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:29.900888 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7548ab46-7548-4a0f-968a-25f7c64cbe5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:49:29.902459 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7548ab46-7548-4a0f-968a-25f7c64cbe5c service nova] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:49:29.911652 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 680/1368] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:49:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:49:31.995490 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2820660-b4e1-4a29-b2fd-18a696fd4e7c None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:31.996615 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2820660-b4e1-4a29-b2fd-18a696fd4e7c None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:31.997120 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2820660-b4e1-4a29-b2fd-18a696fd4e7c None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:32.003438 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 689/1369] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:49:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:49:32.005458 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e6d76-8607-462a-bfc7-101a5014a2c1 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:32.006331 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e6d76-8607-462a-bfc7-101a5014a2c1 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:32.006849 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e6d76-8607-462a-bfc7-101a5014a2c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:32.030529 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-5a1e6d76-8607-462a-bfc7-101a5014a2c1 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:49:32.032203 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-5a1e6d76-8607-462a-bfc7-101a5014a2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:49:32.041677 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 681/1370] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:49:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:49:32.090371 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2820660-b4e1-4a29-b2fd-18a696fd4e7c None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:49:32.105658 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2820660-b4e1-4a29-b2fd-18a696fd4e7c None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:49:32.111242 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71c2bec-a70e-419d-b407-7ab76c8e6638 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4e83df33b64a3ebf28f54e2d94fe63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:49:32.112375 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c2bec-a70e-419d-b407-7ab76c8e6638 service nova] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:32.113119 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c2bec-a70e-419d-b407-7ab76c8e6638 service nova] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:32.113685 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c2bec-a70e-419d-b407-7ab76c8e6638 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:32.125186 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71c2bec-a70e-419d-b407-7ab76c8e6638 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:49:32.126799 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71c2bec-a70e-419d-b407-7ab76c8e6638 service nova] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:49:32.137582 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 690/1371] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:49:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:49:32.739948 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c457e-8896-4cde-9be4-7a3bceacdace None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:32.740205 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c457e-8896-4cde-9be4-7a3bceacdace None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:32.740480 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c457e-8896-4cde-9be4-7a3bceacdace None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:32.742185 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 682/1372] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:49:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:49:32.748235 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cfaa9d-23a5-44c2-9b8b-abb4f78554c0 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:32.748567 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cfaa9d-23a5-44c2-9b8b-abb4f78554c0 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:32.749056 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cfaa9d-23a5-44c2-9b8b-abb4f78554c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:32.767588 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-a5cfaa9d-23a5-44c2-9b8b-abb4f78554c0 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:49:32.768796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-a5cfaa9d-23a5-44c2-9b8b-abb4f78554c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:49:32.777146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 691/1373] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:49:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:49:33.120817 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee67dc63-ff83-45b2-80d0-dbe2060688a4 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:33.120817 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee67dc63-ff83-45b2-80d0-dbe2060688a4 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:33.120817 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee67dc63-ff83-45b2-80d0-dbe2060688a4 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:33.122769 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 683/1374] 10.222.0.26 () {54 vars in 963 bytes} [Thu Jun 23 01:49:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:49:33.126941 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e24fd-2360-47f5-b17c-86f4359fc675 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:33.127515 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e24fd-2360-47f5-b17c-86f4359fc675 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:33.127515 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e24fd-2360-47f5-b17c-86f4359fc675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:33.190183 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-a54e24fd-2360-47f5-b17c-86f4359fc675 None None] MFA Rules not processed for user `dc5bb02f8eb64ac980e3218470b3c1b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:49:33.191738 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-a54e24fd-2360-47f5-b17c-86f4359fc675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:49:33.283160 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 692/1375] 10.222.0.26 () {58 vars in 1060 bytes} [Thu Jun 23 01:49:33 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 01:49:33.292757 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee67dc63-ff83-45b2-80d0-dbe2060688a4 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:49:33.307380 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee67dc63-ff83-45b2-80d0-dbe2060688a4 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:49:33.311271 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5da54d-605d-46a0-aaf5-115ca57c79e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:49:33.311925 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5da54d-605d-46a0-aaf5-115ca57c79e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:33.312205 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5da54d-605d-46a0-aaf5-115ca57c79e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:33.312475 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5da54d-605d-46a0-aaf5-115ca57c79e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:33.322019 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5da54d-605d-46a0-aaf5-115ca57c79e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:49:33.323568 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5da54d-605d-46a0-aaf5-115ca57c79e7 service cinder] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:49:33.334507 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 684/1376] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:49:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:49:34.244987 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af947caf-b434-4ac3-83de-f8b618c0dc81 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:34.245610 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af947caf-b434-4ac3-83de-f8b618c0dc81 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:34.245610 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af947caf-b434-4ac3-83de-f8b618c0dc81 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:34.246847 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 693/1377] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:49:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:49:34.252285 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc6f9ae-b3e0-4872-8eab-ce1e1fadda07 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:34.252727 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc6f9ae-b3e0-4872-8eab-ce1e1fadda07 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:34.252966 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc6f9ae-b3e0-4872-8eab-ce1e1fadda07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:34.271890 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-3bc6f9ae-b3e0-4872-8eab-ce1e1fadda07 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:49:34.273193 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-3bc6f9ae-b3e0-4872-8eab-ce1e1fadda07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:49:34.282126 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 685/1378] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:49:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:49:34.326357 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af947caf-b434-4ac3-83de-f8b618c0dc81 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:49:34.341421 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af947caf-b434-4ac3-83de-f8b618c0dc81 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:49:34.348292 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9dee1a-ebb8-44c9-b7cf-9baabe88db02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4e83df33b64a3ebf28f54e2d94fe63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:49:34.348994 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9dee1a-ebb8-44c9-b7cf-9baabe88db02 service nova] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:34.349255 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9dee1a-ebb8-44c9-b7cf-9baabe88db02 service nova] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:34.349524 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9dee1a-ebb8-44c9-b7cf-9baabe88db02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:34.358478 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9dee1a-ebb8-44c9-b7cf-9baabe88db02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:49:34.360141 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9dee1a-ebb8-44c9-b7cf-9baabe88db02 service nova] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:49:34.364044 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:49:34.372214 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 694/1379] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:49:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:49:36.410198 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d565fd-c5b1-485c-90a6-e0321e91fec2 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:36.411053 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d565fd-c5b1-485c-90a6-e0321e91fec2 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:36.411376 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d565fd-c5b1-485c-90a6-e0321e91fec2 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:36.412757 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 686/1380] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:49:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:49:36.417985 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa90a9-6d4f-4b24-bb90-0a3ccd82ddad None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:36.418413 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa90a9-6d4f-4b24-bb90-0a3ccd82ddad None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:36.418500 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa90a9-6d4f-4b24-bb90-0a3ccd82ddad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:36.438055 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-08aa90a9-6d4f-4b24-bb90-0a3ccd82ddad None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:49:36.439288 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-08aa90a9-6d4f-4b24-bb90-0a3ccd82ddad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:49:36.448490 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 695/1381] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:49:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:49:36.493091 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d565fd-c5b1-485c-90a6-e0321e91fec2 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:49:36.509900 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d565fd-c5b1-485c-90a6-e0321e91fec2 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:49:36.515072 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b755f140-804a-400a-8f00-6d4f045aaeeb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4e83df33b64a3ebf28f54e2d94fe63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:49:36.515984 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755f140-804a-400a-8f00-6d4f045aaeeb service nova] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:36.516411 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755f140-804a-400a-8f00-6d4f045aaeeb service nova] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:36.516663 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b755f140-804a-400a-8f00-6d4f045aaeeb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:36.525615 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b755f140-804a-400a-8f00-6d4f045aaeeb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:49:36.527256 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b755f140-804a-400a-8f00-6d4f045aaeeb service nova] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:49:36.540783 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 687/1382] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:49:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:49:38.591092 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab1043d-6d30-447e-8f67-e213a382c66b None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:38.591643 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab1043d-6d30-447e-8f67-e213a382c66b None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:38.592250 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab1043d-6d30-447e-8f67-e213a382c66b None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:38.593952 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 696/1383] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:49:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:49:38.599567 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c615e55a-5734-41b5-bc00-31a97f5daee3 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:38.600035 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c615e55a-5734-41b5-bc00-31a97f5daee3 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:38.600114 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c615e55a-5734-41b5-bc00-31a97f5daee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:38.619926 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-c615e55a-5734-41b5-bc00-31a97f5daee3 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:49:38.621176 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-c615e55a-5734-41b5-bc00-31a97f5daee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:49:38.630788 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 688/1384] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:49:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:49:38.672935 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab1043d-6d30-447e-8f67-e213a382c66b None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:49:38.687871 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab1043d-6d30-447e-8f67-e213a382c66b None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:49:38.693753 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1f9ef7-9874-42f6-a923-58df290fa600 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4e83df33b64a3ebf28f54e2d94fe63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:49:38.694484 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1f9ef7-9874-42f6-a923-58df290fa600 service nova] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:38.694833 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1f9ef7-9874-42f6-a923-58df290fa600 service nova] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:38.695150 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1f9ef7-9874-42f6-a923-58df290fa600 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:38.704111 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1f9ef7-9874-42f6-a923-58df290fa600 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:49:38.706233 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1f9ef7-9874-42f6-a923-58df290fa600 service nova] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:49:38.716969 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 697/1385] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:49:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 01:49:40.721757 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f3576c-e083-4a21-a968-26b2bbbcbcce None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:40.722369 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f3576c-e083-4a21-a968-26b2bbbcbcce None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:40.722369 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f3576c-e083-4a21-a968-26b2bbbcbcce None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:40.726912 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 689/1386] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:49:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:49:40.729936 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f202bf5b-212c-4afe-946e-f415a392162c None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:40.730394 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f202bf5b-212c-4afe-946e-f415a392162c None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:40.730538 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f202bf5b-212c-4afe-946e-f415a392162c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:40.748530 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-f202bf5b-212c-4afe-946e-f415a392162c None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:49:40.749721 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-f202bf5b-212c-4afe-946e-f415a392162c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:49:40.757506 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 698/1387] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:49:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:49:40.801129 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f3576c-e083-4a21-a968-26b2bbbcbcce None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:49:40.816511 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f3576c-e083-4a21-a968-26b2bbbcbcce None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:49:40.822185 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c94d402-b7ab-4f6a-9721-a4d2c7d0fc29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4e83df33b64a3ebf28f54e2d94fe63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:49:40.822966 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94d402-b7ab-4f6a-9721-a4d2c7d0fc29 service nova] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:40.823289 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94d402-b7ab-4f6a-9721-a4d2c7d0fc29 service nova] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:40.823620 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94d402-b7ab-4f6a-9721-a4d2c7d0fc29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:40.833737 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c94d402-b7ab-4f6a-9721-a4d2c7d0fc29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:49:40.835745 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c94d402-b7ab-4f6a-9721-a4d2c7d0fc29 service nova] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:49:40.847504 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 690/1388] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:49:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:49:43.450211 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafa2dc-3dc9-4c36-95e9-a484bbfa8ac7 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:43.450749 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafa2dc-3dc9-4c36-95e9-a484bbfa8ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:43.451004 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafa2dc-3dc9-4c36-95e9-a484bbfa8ac7 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:43.452942 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 699/1389] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:49:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:49:43.459702 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb57d4-cff8-4880-94b9-c1e148c68c3f None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:43.460149 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb57d4-cff8-4880-94b9-c1e148c68c3f None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:43.460241 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb57d4-cff8-4880-94b9-c1e148c68c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:43.484855 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-78bb57d4-cff8-4880-94b9-c1e148c68c3f None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:49:43.486071 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-78bb57d4-cff8-4880-94b9-c1e148c68c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:49:43.495056 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 691/1390] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:49:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:49:43.543030 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafa2dc-3dc9-4c36-95e9-a484bbfa8ac7 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:49:43.558279 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafa2dc-3dc9-4c36-95e9-a484bbfa8ac7 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:49:43.565071 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c75a505-42b4-4279-9ff1-8e4fbb6ee10e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4e83df33b64a3ebf28f54e2d94fe63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:49:43.565714 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75a505-42b4-4279-9ff1-8e4fbb6ee10e service nova] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:43.565992 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75a505-42b4-4279-9ff1-8e4fbb6ee10e service nova] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:43.566264 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75a505-42b4-4279-9ff1-8e4fbb6ee10e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:43.575343 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c75a505-42b4-4279-9ff1-8e4fbb6ee10e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:49:43.577432 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c75a505-42b4-4279-9ff1-8e4fbb6ee10e service nova] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:49:43.587504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 700/1391] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:49:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:49:46.249649 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903c8c9-bd7d-4ee1-abeb-131704c4e6f3 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:46.250242 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903c8c9-bd7d-4ee1-abeb-131704c4e6f3 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:46.250334 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903c8c9-bd7d-4ee1-abeb-131704c4e6f3 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:46.252122 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 692/1392] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:49:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:49:46.258033 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1683b421-2610-4df2-a672-581c649410c0 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:46.258464 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1683b421-2610-4df2-a672-581c649410c0 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:46.258621 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1683b421-2610-4df2-a672-581c649410c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:46.280786 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-1683b421-2610-4df2-a672-581c649410c0 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:49:46.282005 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-1683b421-2610-4df2-a672-581c649410c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:49:46.290821 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 701/1393] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:49:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:49:46.337817 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e903c8c9-bd7d-4ee1-abeb-131704c4e6f3 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:49:46.353458 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e903c8c9-bd7d-4ee1-abeb-131704c4e6f3 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:49:46.359271 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb16205-d635-4941-a655-0836eb93c426 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4e83df33b64a3ebf28f54e2d94fe63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:49:46.359997 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb16205-d635-4941-a655-0836eb93c426 service nova] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:46.360305 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb16205-d635-4941-a655-0836eb93c426 service nova] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:46.360580 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb16205-d635-4941-a655-0836eb93c426 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:46.369676 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb16205-d635-4941-a655-0836eb93c426 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:49:46.371370 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb16205-d635-4941-a655-0836eb93c426 service nova] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:49:46.381400 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 693/1394] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:49:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:49:48.345373 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01abc8f-b504-4ab9-8d3d-30503febb0b7 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:48.345955 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01abc8f-b504-4ab9-8d3d-30503febb0b7 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:48.345955 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01abc8f-b504-4ab9-8d3d-30503febb0b7 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:48.347347 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 702/1395] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:49:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:49:48.352582 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d5af7-5204-40c1-8381-8020731b4814 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:48.353001 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d5af7-5204-40c1-8381-8020731b4814 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:48.353094 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d5af7-5204-40c1-8381-8020731b4814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:48.372070 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-f04d5af7-5204-40c1-8381-8020731b4814 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:49:48.373425 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-f04d5af7-5204-40c1-8381-8020731b4814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:49:48.382550 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 694/1396] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:49:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:49:48.422217 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b937bc2-0983-4695-b291-6b0948526125 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:48.422506 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b937bc2-0983-4695-b291-6b0948526125 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:48.422751 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b937bc2-0983-4695-b291-6b0948526125 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:48.424180 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 703/1397] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:49:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:49:48.429720 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74923e-a62c-496c-8011-0e3e123f2003 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:48.430036 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74923e-a62c-496c-8011-0e3e123f2003 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:48.430315 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74923e-a62c-496c-8011-0e3e123f2003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:48.441080 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208409f-1238-4d75-980b-b6e97de9c574 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:48.441358 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208409f-1238-4d75-980b-b6e97de9c574 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:48.441764 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208409f-1238-4d75-980b-b6e97de9c574 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:48.443253 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 704/1398] 10.222.0.26 () {54 vars in 962 bytes} [Thu Jun 23 01:49:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:49:48.448024 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d900e41-50a4-4c1f-8ef8-41eac978ade7 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:48.448301 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d900e41-50a4-4c1f-8ef8-41eac978ade7 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:48.448572 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d900e41-50a4-4c1f-8ef8-41eac978ade7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:48.448780 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-3b74923e-a62c-496c-8011-0e3e123f2003 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:49:48.449984 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-3b74923e-a62c-496c-8011-0e3e123f2003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:49:48.457963 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 695/1399] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:49:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 01:49:48.500564 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-0d900e41-50a4-4c1f-8ef8-41eac978ade7 None None] MFA Rules not processed for user `1c081b4cf34146ca8eb197b62a133433`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:49:48.502022 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-0d900e41-50a4-4c1f-8ef8-41eac978ade7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:49:48.502429 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b74923e-a62c-496c-8011-0e3e123f2003 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:49:48.514586 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b74923e-a62c-496c-8011-0e3e123f2003 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:49:48.520428 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285bed66-5dc8-46bc-915f-1ec5edc92ffb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4e83df33b64a3ebf28f54e2d94fe63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:49:48.521214 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285bed66-5dc8-46bc-915f-1ec5edc92ffb service nova] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:48.521502 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285bed66-5dc8-46bc-915f-1ec5edc92ffb service nova] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:48.521769 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285bed66-5dc8-46bc-915f-1ec5edc92ffb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:48.530310 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285bed66-5dc8-46bc-915f-1ec5edc92ffb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:49:48.531963 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285bed66-5dc8-46bc-915f-1ec5edc92ffb service nova] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:49:48.541504 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 696/1400] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:49:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 01:49:48.564951 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 705/1401] 10.222.0.26 () {58 vars in 1059 bytes} [Thu Jun 23 01:49:48 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:49:48.573796 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285bed66-5dc8-46bc-915f-1ec5edc92ffb service nova] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:49:48.583725 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285bed66-5dc8-46bc-915f-1ec5edc92ffb service nova] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:49:48.587390 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b04bde-011f-4e64-bd41-2c8178dc63e7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c081b4cf34146ca8eb197b62a133433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:49:48.588037 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b04bde-011f-4e64-bd41-2c8178dc63e7 service glance] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:48.588317 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b04bde-011f-4e64-bd41-2c8178dc63e7 service glance] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:48.588593 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b04bde-011f-4e64-bd41-2c8178dc63e7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:48.598213 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b04bde-011f-4e64-bd41-2c8178dc63e7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:49:48.599673 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b04bde-011f-4e64-bd41-2c8178dc63e7 service glance] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:49:48.623351 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 697/1402] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:49:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:49:49.320412 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f974da-3967-4f46-9267-093d34cb0215 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:49.320660 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f974da-3967-4f46-9267-093d34cb0215 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:49.320962 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f974da-3967-4f46-9267-093d34cb0215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:49.369576 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-e0f974da-3967-4f46-9267-093d34cb0215 None None] MFA Rules not processed for user `168e6181ad22460990f1f7361b483364`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:49:49.370769 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-e0f974da-3967-4f46-9267-093d34cb0215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:49:49.435495 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:49:49.443636 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 706/1403] 10.222.0.26 () {58 vars in 1009 bytes} [Thu Jun 23 01:49:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:49:49.479985 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e87b4-9cfa-47d4-ae7b-b9713ea4a548 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:49.480482 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e87b4-9cfa-47d4-ae7b-b9713ea4a548 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:49.480596 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e87b4-9cfa-47d4-ae7b-b9713ea4a548 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:49.481912 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 698/1404] 10.222.0.26 () {54 vars in 942 bytes} [Thu Jun 23 01:49:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:49:49.488571 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1157ddc2-9ec0-4724-92a1-036e02153a08 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:49.488973 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1157ddc2-9ec0-4724-92a1-036e02153a08 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:49.489249 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1157ddc2-9ec0-4724-92a1-036e02153a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:49.538463 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-1157ddc2-9ec0-4724-92a1-036e02153a08 None None] MFA Rules not processed for user `28e59688485241c4b6d51f99bbb3d059`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:49:49.539640 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-1157ddc2-9ec0-4724-92a1-036e02153a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:49:49.611510 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 707/1405] 10.222.0.26 () {58 vars in 1039 bytes} [Thu Jun 23 01:49:49 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:49:49.621090 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e87b4-9cfa-47d4-ae7b-b9713ea4a548 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:49:49.634836 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e87b4-9cfa-47d4-ae7b-b9713ea4a548 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:49:49.640279 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a549eec-282e-457a-8689-cb4d0ab8718b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28e59688485241c4b6d51f99bbb3d059', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:49:49.640917 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a549eec-282e-457a-8689-cb4d0ab8718b service swift] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:49.641194 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a549eec-282e-457a-8689-cb4d0ab8718b service swift] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:49.641469 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a549eec-282e-457a-8689-cb4d0ab8718b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:49.649863 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a549eec-282e-457a-8689-cb4d0ab8718b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:49:49.651479 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a549eec-282e-457a-8689-cb4d0ab8718b service swift] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:49:49.659825 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 699/1406] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Jun 23 01:49:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 01:49:50.607895 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935060d6-e5e4-4cec-8c58-f206986957b0 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:50.608768 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935060d6-e5e4-4cec-8c58-f206986957b0 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:50.609049 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935060d6-e5e4-4cec-8c58-f206986957b0 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:50.610520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 708/1407] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:49:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:49:50.615972 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd86b4e2-2a3b-4b5c-be80-797629498c0f None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:50.616382 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd86b4e2-2a3b-4b5c-be80-797629498c0f None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:50.616555 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd86b4e2-2a3b-4b5c-be80-797629498c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:50.622769 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=76 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:49:50.636679 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-fd86b4e2-2a3b-4b5c-be80-797629498c0f None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:49:50.637880 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-fd86b4e2-2a3b-4b5c-be80-797629498c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:49:50.646075 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 700/1408] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:49:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:49:50.690816 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935060d6-e5e4-4cec-8c58-f206986957b0 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:49:50.706877 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935060d6-e5e4-4cec-8c58-f206986957b0 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:49:50.712545 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fecdeb9-16b3-4076-b0b0-8c839e40ef14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4e83df33b64a3ebf28f54e2d94fe63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:49:50.713177 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fecdeb9-16b3-4076-b0b0-8c839e40ef14 service nova] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:49:50.713461 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fecdeb9-16b3-4076-b0b0-8c839e40ef14 service nova] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:49:50.713737 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fecdeb9-16b3-4076-b0b0-8c839e40ef14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:49:50.723635 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fecdeb9-16b3-4076-b0b0-8c839e40ef14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:49:50.725245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fecdeb9-16b3-4076-b0b0-8c839e40ef14 service nova] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:49:50.737231 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 709/1409] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:49:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:50:00.631555 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=2 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:50:00.731672 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=50 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:50:11.490200 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aba98f-9366-485d-a735-2f215069239c None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:11.490754 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aba98f-9366-485d-a735-2f215069239c None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:11.490754 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aba98f-9366-485d-a735-2f215069239c None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:11.492068 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 701/1410] 10.222.0.26 () {54 vars in 918 bytes} [Thu Jun 23 01:50:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:50:11.497369 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d570a849-3d7c-468e-86e8-6eadc086f28f None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:11.497845 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d570a849-3d7c-468e-86e8-6eadc086f28f None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:11.497954 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d570a849-3d7c-468e-86e8-6eadc086f28f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:11.517467 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-d570a849-3d7c-468e-86e8-6eadc086f28f None None] MFA Rules not processed for user `e5bf3baee1b843fa89b65e5485f6bebb`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:50:11.518681 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-d570a849-3d7c-468e-86e8-6eadc086f28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:50:11.528892 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 710/1411] 10.222.0.26 () {58 vars in 1015 bytes} [Thu Jun 23 01:50:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:50:11.543411 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5987f652-806b-486c-bbad-bad92e367a64 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:11.543696 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5987f652-806b-486c-bbad-bad92e367a64 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:11.543969 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5987f652-806b-486c-bbad-bad92e367a64 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:11.545347 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 702/1412] 10.222.0.26 () {54 vars in 918 bytes} [Thu Jun 23 01:50:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:50:11.550048 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d21027-d5ba-451e-ace5-e1ac1f06fee4 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:11.550328 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d21027-d5ba-451e-ace5-e1ac1f06fee4 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:11.550601 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d21027-d5ba-451e-ace5-e1ac1f06fee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:11.569533 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-f4d21027-d5ba-451e-ace5-e1ac1f06fee4 None None] MFA Rules not processed for user `ae4e83df33b64a3ebf28f54e2d94fe63`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:50:11.570740 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-f4d21027-d5ba-451e-ace5-e1ac1f06fee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:50:11.581770 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 711/1413] 10.222.0.26 () {58 vars in 1015 bytes} [Thu Jun 23 01:50:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 01:50:14.067121 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efbe2df-6dd4-45ad-aceb-7c4ecec66eb8 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:14.067733 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efbe2df-6dd4-45ad-aceb-7c4ecec66eb8 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:14.067733 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efbe2df-6dd4-45ad-aceb-7c4ecec66eb8 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:14.069067 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 703/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 01:50:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:50:14.074425 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da5b08-b0b4-425c-aeff-dbad074d674a None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:14.074866 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da5b08-b0b4-425c-aeff-dbad074d674a None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:14.074970 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da5b08-b0b4-425c-aeff-dbad074d674a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:14.094923 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-60da5b08-b0b4-425c-aeff-dbad074d674a None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:50:14.096520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-60da5b08-b0b4-425c-aeff-dbad074d674a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:50:14.104730 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 712/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 01:50:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:50:14.152159 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efbe2df-6dd4-45ad-aceb-7c4ecec66eb8 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:50:14.171542 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efbe2df-6dd4-45ad-aceb-7c4ecec66eb8 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:50:14.177279 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052d562e-f268-41d5-9616-9450fbb147f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4e83df33b64a3ebf28f54e2d94fe63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:50:14.177916 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d562e-f268-41d5-9616-9450fbb147f8 service nova] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:14.178518 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d562e-f268-41d5-9616-9450fbb147f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:14.178792 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d562e-f268-41d5-9616-9450fbb147f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:14.187165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052d562e-f268-41d5-9616-9450fbb147f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:50:14.188801 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052d562e-f268-41d5-9616-9450fbb147f8 service nova] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:50:14.198212 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 704/1416] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:50:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:50:15.133143 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dace42c-2237-4474-9786-128b491fc927 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:15.134071 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dace42c-2237-4474-9786-128b491fc927 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:15.134071 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dace42c-2237-4474-9786-128b491fc927 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:15.136397 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 713/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 01:50:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:50:15.141879 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b2d03-4ab2-4295-b5ed-26a613c3b1da None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:15.142413 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b2d03-4ab2-4295-b5ed-26a613c3b1da None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:15.142413 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b2d03-4ab2-4295-b5ed-26a613c3b1da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:15.161508 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-012b2d03-4ab2-4295-b5ed-26a613c3b1da None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:50:15.162722 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-012b2d03-4ab2-4295-b5ed-26a613c3b1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:50:15.171773 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 705/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 01:50:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:50:15.332245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dace42c-2237-4474-9786-128b491fc927 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:50:15.347711 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dace42c-2237-4474-9786-128b491fc927 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:50:15.353748 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445aa508-abe7-42dd-b9a3-a4b654d22af3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4e83df33b64a3ebf28f54e2d94fe63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:50:15.354383 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445aa508-abe7-42dd-b9a3-a4b654d22af3 service nova] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:15.354656 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445aa508-abe7-42dd-b9a3-a4b654d22af3 service nova] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:15.354929 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445aa508-abe7-42dd-b9a3-a4b654d22af3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:15.363536 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445aa508-abe7-42dd-b9a3-a4b654d22af3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:50:15.365119 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445aa508-abe7-42dd-b9a3-a4b654d22af3 service nova] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:50:15.374746 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 714/1419] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:50:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:50:16.254735 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e402b-560c-4965-a26b-ba9b3b614141 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:16.255328 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e402b-560c-4965-a26b-ba9b3b614141 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:16.255328 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e402b-560c-4965-a26b-ba9b3b614141 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:16.256732 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 706/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 01:50:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:50:16.263848 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d643f5f-92cc-4ec1-be59-ebc4dc06b25d None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:16.264344 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d643f5f-92cc-4ec1-be59-ebc4dc06b25d None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:16.264441 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d643f5f-92cc-4ec1-be59-ebc4dc06b25d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:16.284237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-4d643f5f-92cc-4ec1-be59-ebc4dc06b25d None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:50:16.285445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-4d643f5f-92cc-4ec1-be59-ebc4dc06b25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:50:16.296421 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 715/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 01:50:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:50:16.450206 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e402b-560c-4965-a26b-ba9b3b614141 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:50:16.466958 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e402b-560c-4965-a26b-ba9b3b614141 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:50:16.473920 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c239b0ec-199b-44bd-909d-e95e2d86f5cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4e83df33b64a3ebf28f54e2d94fe63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:50:16.474526 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239b0ec-199b-44bd-909d-e95e2d86f5cd service nova] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:16.474810 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239b0ec-199b-44bd-909d-e95e2d86f5cd service nova] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:16.475085 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239b0ec-199b-44bd-909d-e95e2d86f5cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:16.483647 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c239b0ec-199b-44bd-909d-e95e2d86f5cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:50:16.485235 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c239b0ec-199b-44bd-909d-e95e2d86f5cd service nova] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:50:16.494811 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 707/1422] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:50:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:50:17.370497 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a098c9-fbda-4887-9381-3ac65cdd5317 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:17.371082 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a098c9-fbda-4887-9381-3ac65cdd5317 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:17.371082 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a098c9-fbda-4887-9381-3ac65cdd5317 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:17.372461 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 716/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 01:50:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:50:17.377790 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a24c4-6a38-47a0-bfe5-3d610205d5e7 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:17.378203 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a24c4-6a38-47a0-bfe5-3d610205d5e7 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:17.378386 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a24c4-6a38-47a0-bfe5-3d610205d5e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:17.398383 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-2c6a24c4-6a38-47a0-bfe5-3d610205d5e7 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:50:17.399679 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-2c6a24c4-6a38-47a0-bfe5-3d610205d5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:50:17.407970 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 708/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 01:50:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:50:17.560917 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a098c9-fbda-4887-9381-3ac65cdd5317 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:50:17.577502 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a098c9-fbda-4887-9381-3ac65cdd5317 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:50:17.584419 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc99df8-c6ee-4ece-a27d-7099d98c08bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4e83df33b64a3ebf28f54e2d94fe63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:50:17.585048 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc99df8-c6ee-4ece-a27d-7099d98c08bb service nova] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:17.585329 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc99df8-c6ee-4ece-a27d-7099d98c08bb service nova] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:17.585575 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc99df8-c6ee-4ece-a27d-7099d98c08bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:17.594716 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc99df8-c6ee-4ece-a27d-7099d98c08bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:50:17.596366 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc99df8-c6ee-4ece-a27d-7099d98c08bb service nova] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:50:17.608719 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 717/1425] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:50:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:50:18.471063 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41bb8cd-5750-4a13-81ec-ee5bc6e40d92 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:18.471909 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41bb8cd-5750-4a13-81ec-ee5bc6e40d92 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:18.471909 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41bb8cd-5750-4a13-81ec-ee5bc6e40d92 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:18.474050 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 709/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 01:50:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:50:18.480190 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e4741-5b6c-46ae-b202-5d817b02e64d None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:18.480639 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e4741-5b6c-46ae-b202-5d817b02e64d None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:18.480799 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e4741-5b6c-46ae-b202-5d817b02e64d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:18.505460 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-420e4741-5b6c-46ae-b202-5d817b02e64d None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:50:18.506685 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-420e4741-5b6c-46ae-b202-5d817b02e64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:50:18.515239 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 718/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 01:50:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:50:18.670419 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41bb8cd-5750-4a13-81ec-ee5bc6e40d92 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:50:18.696909 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41bb8cd-5750-4a13-81ec-ee5bc6e40d92 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:50:18.705164 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2196ebe8-66ef-47ff-8848-d93044cc038c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4e83df33b64a3ebf28f54e2d94fe63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:50:18.705806 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196ebe8-66ef-47ff-8848-d93044cc038c service nova] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:18.706084 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196ebe8-66ef-47ff-8848-d93044cc038c service nova] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:18.706362 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196ebe8-66ef-47ff-8848-d93044cc038c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:18.715089 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2196ebe8-66ef-47ff-8848-d93044cc038c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:50:18.716715 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2196ebe8-66ef-47ff-8848-d93044cc038c service nova] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:50:18.726433 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 710/1428] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:50:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:50:19.531072 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18cda5-1627-4e55-8ee2-ae182fe65c10 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:19.531998 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18cda5-1627-4e55-8ee2-ae182fe65c10 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:19.531998 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18cda5-1627-4e55-8ee2-ae182fe65c10 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:19.534157 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 719/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 01:50:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:50:19.542508 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a261751-5478-45bb-99a6-1d7e5bd6ec6b None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:19.542929 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a261751-5478-45bb-99a6-1d7e5bd6ec6b None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:19.543099 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a261751-5478-45bb-99a6-1d7e5bd6ec6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:19.563160 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-4a261751-5478-45bb-99a6-1d7e5bd6ec6b None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:50:19.564406 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-4a261751-5478-45bb-99a6-1d7e5bd6ec6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:50:19.573444 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 711/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 01:50:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:50:19.726766 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a18cda5-1627-4e55-8ee2-ae182fe65c10 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:50:19.745609 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a18cda5-1627-4e55-8ee2-ae182fe65c10 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:50:19.751889 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c9b4cd-2bef-45cc-84a9-f68be3d2835c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4e83df33b64a3ebf28f54e2d94fe63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:50:19.752518 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c9b4cd-2bef-45cc-84a9-f68be3d2835c service nova] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:19.752792 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c9b4cd-2bef-45cc-84a9-f68be3d2835c service nova] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:19.753063 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c9b4cd-2bef-45cc-84a9-f68be3d2835c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:19.761816 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c9b4cd-2bef-45cc-84a9-f68be3d2835c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:50:19.763427 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c9b4cd-2bef-45cc-84a9-f68be3d2835c service nova] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:50:19.772866 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 720/1431] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:50:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:50:20.583278 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d8919-5ea6-4c3f-8a0c-1277873f9206 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:20.583833 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d8919-5ea6-4c3f-8a0c-1277873f9206 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:20.583833 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d8919-5ea6-4c3f-8a0c-1277873f9206 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:20.585653 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 712/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 01:50:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:50:20.592948 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f79eb7-239d-448d-9335-18b745f9d19e None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:20.593353 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f79eb7-239d-448d-9335-18b745f9d19e None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:20.593503 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f79eb7-239d-448d-9335-18b745f9d19e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:20.612192 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-31f79eb7-239d-448d-9335-18b745f9d19e None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:50:20.613407 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-31f79eb7-239d-448d-9335-18b745f9d19e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:50:20.621723 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 721/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 01:50:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:50:20.774624 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d8919-5ea6-4c3f-8a0c-1277873f9206 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:50:20.792666 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d8919-5ea6-4c3f-8a0c-1277873f9206 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:50:20.799363 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35007e41-51f7-4481-9d90-0768cd1864a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4e83df33b64a3ebf28f54e2d94fe63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:50:20.800024 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35007e41-51f7-4481-9d90-0768cd1864a0 service nova] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:20.800304 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35007e41-51f7-4481-9d90-0768cd1864a0 service nova] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:20.800584 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35007e41-51f7-4481-9d90-0768cd1864a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:20.809430 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35007e41-51f7-4481-9d90-0768cd1864a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:50:20.811096 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35007e41-51f7-4481-9d90-0768cd1864a0 service nova] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:50:20.820723 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 713/1434] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:50:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:50:21.626598 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c443ad-e81f-4682-885a-b6d23408d818 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:21.627256 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c443ad-e81f-4682-885a-b6d23408d818 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:21.627256 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c443ad-e81f-4682-885a-b6d23408d818 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:21.628486 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 722/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 01:50:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:50:21.635153 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b4e98-dc18-4d85-bf0c-0eab0da4bf16 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:21.635932 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b4e98-dc18-4d85-bf0c-0eab0da4bf16 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:21.636101 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b4e98-dc18-4d85-bf0c-0eab0da4bf16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:21.656593 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-1d8b4e98-dc18-4d85-bf0c-0eab0da4bf16 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:50:21.657802 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-1d8b4e98-dc18-4d85-bf0c-0eab0da4bf16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:50:21.667127 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 714/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 01:50:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:50:21.818667 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c443ad-e81f-4682-885a-b6d23408d818 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:50:21.834864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c443ad-e81f-4682-885a-b6d23408d818 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:50:21.840901 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ce4a9-e6df-4fd8-a5f4-eae8d6d78b52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4e83df33b64a3ebf28f54e2d94fe63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:50:21.841555 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ce4a9-e6df-4fd8-a5f4-eae8d6d78b52 service nova] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:21.841832 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ce4a9-e6df-4fd8-a5f4-eae8d6d78b52 service nova] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:21.842102 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ce4a9-e6df-4fd8-a5f4-eae8d6d78b52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:21.850470 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101ce4a9-e6df-4fd8-a5f4-eae8d6d78b52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:50:21.852105 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101ce4a9-e6df-4fd8-a5f4-eae8d6d78b52 service nova] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:50:21.861630 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 723/1437] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:50:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:50:22.659719 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0574b1f6-be96-4fe7-a791-0c0cd36c1eed None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:22.660539 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0574b1f6-be96-4fe7-a791-0c0cd36c1eed None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:22.660539 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0574b1f6-be96-4fe7-a791-0c0cd36c1eed None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:22.662695 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 715/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 01:50:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:50:22.668686 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60b5b4-4561-45bc-98b6-a7de3f63a8bc None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:22.669423 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60b5b4-4561-45bc-98b6-a7de3f63a8bc None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:22.669715 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60b5b4-4561-45bc-98b6-a7de3f63a8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:22.689105 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-de60b5b4-4561-45bc-98b6-a7de3f63a8bc None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:50:22.690320 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-de60b5b4-4561-45bc-98b6-a7de3f63a8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:50:22.699276 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 724/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 01:50:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:50:22.850257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0574b1f6-be96-4fe7-a791-0c0cd36c1eed None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:50:22.873747 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0574b1f6-be96-4fe7-a791-0c0cd36c1eed None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:50:22.879548 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1970bd06-f1ad-4e1a-b86b-9faac1eca12b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4e83df33b64a3ebf28f54e2d94fe63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:50:22.880180 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970bd06-f1ad-4e1a-b86b-9faac1eca12b service nova] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:22.880460 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970bd06-f1ad-4e1a-b86b-9faac1eca12b service nova] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:22.880734 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970bd06-f1ad-4e1a-b86b-9faac1eca12b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:22.889250 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1970bd06-f1ad-4e1a-b86b-9faac1eca12b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:50:22.890829 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1970bd06-f1ad-4e1a-b86b-9faac1eca12b service nova] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:50:22.900757 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 716/1440] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:50:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:50:24.816975 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b2b35-8d91-4d85-8586-c1a738e9791b None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:24.817582 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b2b35-8d91-4d85-8586-c1a738e9791b None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:24.817582 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b2b35-8d91-4d85-8586-c1a738e9791b None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:24.818918 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 725/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 01:50:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 01:50:24.825357 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7175a2-3f4a-493f-b936-c85d3e9289ac None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:24.826025 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7175a2-3f4a-493f-b936-c85d3e9289ac None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:24.826216 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7175a2-3f4a-493f-b936-c85d3e9289ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:24.844531 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-8c7175a2-3f4a-493f-b936-c85d3e9289ac None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:50:24.845739 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-8c7175a2-3f4a-493f-b936-c85d3e9289ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:50:24.854366 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 717/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 01:50:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:50:24.892198 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478b2b35-8d91-4d85-8586-c1a738e9791b None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:50:24.894263 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-478b2b35-8d91-4d85-8586-c1a738e9791b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:50:24.908810 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478b2b35-8d91-4d85-8586-c1a738e9791b None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:50:24.912812 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454aa653-c78f-47ce-a170-8a327ce314e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:50:24.913438 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454aa653-c78f-47ce-a170-8a327ce314e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:24.913719 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454aa653-c78f-47ce-a170-8a327ce314e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:24.913991 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454aa653-c78f-47ce-a170-8a327ce314e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:24.923150 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454aa653-c78f-47ce-a170-8a327ce314e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:50:24.924622 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454aa653-c78f-47ce-a170-8a327ce314e1 service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:50:24.934208 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 726/1443] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:50:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 01:50:27.089953 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33bbcab-bb04-4408-9450-5a17fc85e9d6 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:27.090562 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33bbcab-bb04-4408-9450-5a17fc85e9d6 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:27.090562 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33bbcab-bb04-4408-9450-5a17fc85e9d6 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:27.092664 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 718/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 01:50:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:50:27.100975 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b2ec1-ef37-42ac-acda-67da9e35478e None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:27.101398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b2ec1-ef37-42ac-acda-67da9e35478e None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:27.101539 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b2ec1-ef37-42ac-acda-67da9e35478e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:27.122576 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-818b2ec1-ef37-42ac-acda-67da9e35478e None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:50:27.123814 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-818b2ec1-ef37-42ac-acda-67da9e35478e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:50:27.132582 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 727/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 01:50:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:50:27.170241 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33bbcab-bb04-4408-9450-5a17fc85e9d6 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:50:27.182432 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33bbcab-bb04-4408-9450-5a17fc85e9d6 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:50:27.186257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae64a7c-8732-4c12-9a97-51d9aec8ea38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:50:27.186564 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae64a7c-8732-4c12-9a97-51d9aec8ea38 service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:27.186848 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae64a7c-8732-4c12-9a97-51d9aec8ea38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:27.187125 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae64a7c-8732-4c12-9a97-51d9aec8ea38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:27.196909 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae64a7c-8732-4c12-9a97-51d9aec8ea38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:50:27.198361 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae64a7c-8732-4c12-9a97-51d9aec8ea38 service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:50:27.208379 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 719/1446] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:50:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:50:29.610311 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b08499-197b-4614-a620-e968712e13eb None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:29.610954 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b08499-197b-4614-a620-e968712e13eb None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:29.610954 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b08499-197b-4614-a620-e968712e13eb None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:29.612223 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 728/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 01:50:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:50:29.618530 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dab610-b3b5-4aeb-9b22-582636d7de01 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:29.618982 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dab610-b3b5-4aeb-9b22-582636d7de01 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:29.619096 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dab610-b3b5-4aeb-9b22-582636d7de01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:29.639483 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-d0dab610-b3b5-4aeb-9b22-582636d7de01 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:50:29.640687 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-d0dab610-b3b5-4aeb-9b22-582636d7de01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:50:29.650468 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 720/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 01:50:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:50:29.690577 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b08499-197b-4614-a620-e968712e13eb None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:50:29.709343 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b08499-197b-4614-a620-e968712e13eb None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:50:29.715962 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3242ca-987a-449b-966e-6f9262e6cb95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:50:29.716745 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3242ca-987a-449b-966e-6f9262e6cb95 service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:29.717149 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3242ca-987a-449b-966e-6f9262e6cb95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:29.717428 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3242ca-987a-449b-966e-6f9262e6cb95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:29.727821 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3242ca-987a-449b-966e-6f9262e6cb95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:50:29.729277 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3242ca-987a-449b-966e-6f9262e6cb95 service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:50:29.739145 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 729/1449] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:50:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:50:31.867499 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37341c-ea99-46a8-8433-4a3b445d6c8e None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:31.867499 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37341c-ea99-46a8-8433-4a3b445d6c8e None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:31.867499 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37341c-ea99-46a8-8433-4a3b445d6c8e None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:31.868693 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 721/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 01:50:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:50:31.875776 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03151ab4-4a2c-42f7-8839-83149d68fa57 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:31.876200 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03151ab4-4a2c-42f7-8839-83149d68fa57 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:31.876314 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03151ab4-4a2c-42f7-8839-83149d68fa57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:31.904628 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-03151ab4-4a2c-42f7-8839-83149d68fa57 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:50:31.905852 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-03151ab4-4a2c-42f7-8839-83149d68fa57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:50:31.914169 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 730/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 01:50:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:50:31.952890 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c37341c-ea99-46a8-8433-4a3b445d6c8e None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:50:31.966213 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c37341c-ea99-46a8-8433-4a3b445d6c8e None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:50:31.970788 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b6ad7-9046-4780-82c4-add887b0119f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:50:31.971478 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b6ad7-9046-4780-82c4-add887b0119f service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:31.971766 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b6ad7-9046-4780-82c4-add887b0119f service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:31.972055 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b6ad7-9046-4780-82c4-add887b0119f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:31.982521 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156b6ad7-9046-4780-82c4-add887b0119f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:50:31.984002 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156b6ad7-9046-4780-82c4-add887b0119f service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:50:31.993906 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 722/1452] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:50:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:50:34.122799 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7184df0-8ca8-4d29-b7c6-433d4599cb55 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:34.123463 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7184df0-8ca8-4d29-b7c6-433d4599cb55 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:34.123463 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7184df0-8ca8-4d29-b7c6-433d4599cb55 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:34.125029 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 731/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 01:50:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:50:34.131943 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f807e-6bdf-402c-85f3-29060320ad98 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:34.132736 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f807e-6bdf-402c-85f3-29060320ad98 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:34.132736 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f807e-6bdf-402c-85f3-29060320ad98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:34.160971 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-7c6f807e-6bdf-402c-85f3-29060320ad98 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:50:34.162189 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-7c6f807e-6bdf-402c-85f3-29060320ad98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:50:34.172033 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 723/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 01:50:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:50:34.217748 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7184df0-8ca8-4d29-b7c6-433d4599cb55 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:50:34.229883 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7184df0-8ca8-4d29-b7c6-433d4599cb55 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:50:34.235067 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c881b-5102-4b30-a96a-e6a4d7917527 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:50:34.236110 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c881b-5102-4b30-a96a-e6a4d7917527 service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:34.236618 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c881b-5102-4b30-a96a-e6a4d7917527 service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:34.237000 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c881b-5102-4b30-a96a-e6a4d7917527 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:34.248227 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6c881b-5102-4b30-a96a-e6a4d7917527 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:50:34.249679 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6c881b-5102-4b30-a96a-e6a4d7917527 service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:50:34.260673 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 732/1455] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:50:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:50:36.369181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436cfcc0-0d3a-4522-8fdd-d201292f4d65 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:36.369743 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436cfcc0-0d3a-4522-8fdd-d201292f4d65 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:36.369743 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436cfcc0-0d3a-4522-8fdd-d201292f4d65 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:36.371091 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 724/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 01:50:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:50:36.377576 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320e54c-474b-40ea-9b05-1362deb1d8df None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:36.378360 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320e54c-474b-40ea-9b05-1362deb1d8df None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:36.378518 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320e54c-474b-40ea-9b05-1362deb1d8df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:36.407022 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-7320e54c-474b-40ea-9b05-1362deb1d8df None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:50:36.408275 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-7320e54c-474b-40ea-9b05-1362deb1d8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:50:36.418670 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 733/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 01:50:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:50:36.457394 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436cfcc0-0d3a-4522-8fdd-d201292f4d65 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:50:36.472335 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436cfcc0-0d3a-4522-8fdd-d201292f4d65 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:50:36.476709 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f53c15-ec94-4fb4-ad3c-2fab9efa53d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:50:36.477343 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f53c15-ec94-4fb4-ad3c-2fab9efa53d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:36.477627 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f53c15-ec94-4fb4-ad3c-2fab9efa53d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:36.477894 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f53c15-ec94-4fb4-ad3c-2fab9efa53d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:36.487236 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f53c15-ec94-4fb4-ad3c-2fab9efa53d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:50:36.488671 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f53c15-ec94-4fb4-ad3c-2fab9efa53d2 service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:50:36.501385 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 725/1458] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:50:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:50:38.622109 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04756c83-344e-4b8f-833a-e5bf0406746f None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:38.622709 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04756c83-344e-4b8f-833a-e5bf0406746f None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:38.622709 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04756c83-344e-4b8f-833a-e5bf0406746f None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:38.624073 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 734/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 01:50:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:50:38.630499 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699102b6-0d64-40c2-abc2-f266e0b40f72 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:38.630958 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699102b6-0d64-40c2-abc2-f266e0b40f72 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:38.631065 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699102b6-0d64-40c2-abc2-f266e0b40f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:38.658744 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-699102b6-0d64-40c2-abc2-f266e0b40f72 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:50:38.659995 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-699102b6-0d64-40c2-abc2-f266e0b40f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:50:38.669435 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 726/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 01:50:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:50:38.707853 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04756c83-344e-4b8f-833a-e5bf0406746f None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:50:38.721351 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04756c83-344e-4b8f-833a-e5bf0406746f None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:50:38.726030 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087ab28c-7d3d-4b33-9700-1ca9f53b0a17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:50:38.726758 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ab28c-7d3d-4b33-9700-1ca9f53b0a17 service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:38.727033 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ab28c-7d3d-4b33-9700-1ca9f53b0a17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:38.727328 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ab28c-7d3d-4b33-9700-1ca9f53b0a17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:38.737438 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087ab28c-7d3d-4b33-9700-1ca9f53b0a17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:50:38.738945 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087ab28c-7d3d-4b33-9700-1ca9f53b0a17 service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:50:38.748930 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 735/1461] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:50:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:50:40.774271 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58ff33-e715-4565-b6ee-81a12b5f61af None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:40.774850 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58ff33-e715-4565-b6ee-81a12b5f61af None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:40.774850 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58ff33-e715-4565-b6ee-81a12b5f61af None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:40.777096 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 727/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 01:50:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:50:40.784083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4d43f-c0cd-4969-a2ff-890b8c2c3e4f None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:40.784504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4d43f-c0cd-4969-a2ff-890b8c2c3e4f None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:40.784711 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4d43f-c0cd-4969-a2ff-890b8c2c3e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:40.795057 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=79 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:50:40.813737 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-abe4d43f-c0cd-4969-a2ff-890b8c2c3e4f None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:50:40.814941 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-abe4d43f-c0cd-4969-a2ff-890b8c2c3e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:50:40.824245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 736/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 01:50:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:50:40.862225 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd58ff33-e715-4565-b6ee-81a12b5f61af None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:50:40.870040 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=73 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:50:40.877300 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd58ff33-e715-4565-b6ee-81a12b5f61af None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:50:40.881285 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d198c80-b224-41d4-93dc-65c2fae995bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:50:40.881936 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d198c80-b224-41d4-93dc-65c2fae995bf service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:40.882225 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d198c80-b224-41d4-93dc-65c2fae995bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:40.882487 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d198c80-b224-41d4-93dc-65c2fae995bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:40.891996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d198c80-b224-41d4-93dc-65c2fae995bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:50:40.893432 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d198c80-b224-41d4-93dc-65c2fae995bf service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:50:40.903080 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 728/1464] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:50:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:50:42.934003 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a851c-4f8c-4a93-8d84-06a7fbe15c48 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:42.934570 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a851c-4f8c-4a93-8d84-06a7fbe15c48 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:42.934570 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1a851c-4f8c-4a93-8d84-06a7fbe15c48 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:42.936007 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 737/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 01:50:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:50:42.942414 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827eda4d-e0df-42c2-9f73-500c48614c82 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:42.943141 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827eda4d-e0df-42c2-9f73-500c48614c82 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:42.943328 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827eda4d-e0df-42c2-9f73-500c48614c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:42.964673 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-827eda4d-e0df-42c2-9f73-500c48614c82 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:50:42.965878 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-827eda4d-e0df-42c2-9f73-500c48614c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:50:42.974627 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 729/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 01:50:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:50:43.015444 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1a851c-4f8c-4a93-8d84-06a7fbe15c48 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:50:43.032220 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1a851c-4f8c-4a93-8d84-06a7fbe15c48 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:50:43.039607 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc13eb1-b996-4b12-8a28-b37ac3425ac3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:50:43.040236 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc13eb1-b996-4b12-8a28-b37ac3425ac3 service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:43.040509 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc13eb1-b996-4b12-8a28-b37ac3425ac3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:43.040774 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc13eb1-b996-4b12-8a28-b37ac3425ac3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:43.051733 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc13eb1-b996-4b12-8a28-b37ac3425ac3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:50:43.053562 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc13eb1-b996-4b12-8a28-b37ac3425ac3 service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:50:43.063783 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 738/1467] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:50:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:50:45.106206 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a830e-d09b-4be0-9c89-5f8bac8c5af7 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:45.106860 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a830e-d09b-4be0-9c89-5f8bac8c5af7 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:45.106860 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a830e-d09b-4be0-9c89-5f8bac8c5af7 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:45.108263 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 730/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 01:50:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:50:45.113892 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43435b-a542-4020-a3d9-d5596dc7d400 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:45.114340 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43435b-a542-4020-a3d9-d5596dc7d400 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:45.114444 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43435b-a542-4020-a3d9-d5596dc7d400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:45.137994 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-8e43435b-a542-4020-a3d9-d5596dc7d400 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:50:45.139246 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-8e43435b-a542-4020-a3d9-d5596dc7d400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:50:45.147754 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 739/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 01:50:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:50:45.187537 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82a830e-d09b-4be0-9c89-5f8bac8c5af7 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:50:45.203626 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82a830e-d09b-4be0-9c89-5f8bac8c5af7 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:50:45.208150 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f216f6-1b53-487d-a7f6-cc1f6e9fa4bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:50:45.208794 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f216f6-1b53-487d-a7f6-cc1f6e9fa4bf service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:50:45.209077 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f216f6-1b53-487d-a7f6-cc1f6e9fa4bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:50:45.209353 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f216f6-1b53-487d-a7f6-cc1f6e9fa4bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:50:45.218654 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f216f6-1b53-487d-a7f6-cc1f6e9fa4bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:50:45.220137 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f216f6-1b53-487d-a7f6-cc1f6e9fa4bf service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:50:45.229494 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 731/1470] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:50:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:50:55.130895 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=11 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:50:55.225291 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=14 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:51:04.915841 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956c4b0-07ed-410f-a2e7-56e66080c0ca None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:51:04.916451 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956c4b0-07ed-410f-a2e7-56e66080c0ca None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:51:04.916451 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956c4b0-07ed-410f-a2e7-56e66080c0ca None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:51:04.918433 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 740/1471] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:51:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:51:04.926363 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba5df6-b206-4050-b31b-f25253191d20 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:51:04.926862 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba5df6-b206-4050-b31b-f25253191d20 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:51:04.926862 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba5df6-b206-4050-b31b-f25253191d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:51:04.946706 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-98ba5df6-b206-4050-b31b-f25253191d20 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:51:04.948020 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-98ba5df6-b206-4050-b31b-f25253191d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:51:04.957269 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 732/1472] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:51:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 01:51:05.006459 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31646a-1c5f-470c-9c96-7c2b08dd25de None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:51:05.006741 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31646a-1c5f-470c-9c96-7c2b08dd25de None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:51:05.007022 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31646a-1c5f-470c-9c96-7c2b08dd25de None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:51:05.008428 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 741/1473] 10.222.0.26 () {54 vars in 962 bytes} [Thu Jun 23 01:51:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:51:05.014202 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aeae03-529d-4072-9d7c-e9c1347de3b1 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:51:05.014588 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aeae03-529d-4072-9d7c-e9c1347de3b1 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:51:05.014943 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aeae03-529d-4072-9d7c-e9c1347de3b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:51:05.038471 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-a8aeae03-529d-4072-9d7c-e9c1347de3b1 None None] MFA Rules not processed for user `1c081b4cf34146ca8eb197b62a133433`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:51:05.039737 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-a8aeae03-529d-4072-9d7c-e9c1347de3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:51:05.047704 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 733/1474] 10.222.0.26 () {58 vars in 1059 bytes} [Thu Jun 23 01:51:05 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:51:05.055720 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc31646a-1c5f-470c-9c96-7c2b08dd25de None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:51:05.067767 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc31646a-1c5f-470c-9c96-7c2b08dd25de None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:51:05.071438 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5781c1f5-9185-4b7e-ad4f-093c89e5b80f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c081b4cf34146ca8eb197b62a133433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:51:05.072081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781c1f5-9185-4b7e-ad4f-093c89e5b80f service glance] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:51:05.072365 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781c1f5-9185-4b7e-ad4f-093c89e5b80f service glance] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:51:05.072639 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781c1f5-9185-4b7e-ad4f-093c89e5b80f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:51:05.081807 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5781c1f5-9185-4b7e-ad4f-093c89e5b80f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:51:05.083244 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5781c1f5-9185-4b7e-ad4f-093c89e5b80f service glance] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:51:05.092775 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 742/1475] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:51:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:51:07.209768 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac75eb4-a053-4506-a5a0-9f228b0154aa None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:51:07.210444 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac75eb4-a053-4506-a5a0-9f228b0154aa None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:51:07.210631 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac75eb4-a053-4506-a5a0-9f228b0154aa None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:51:07.212891 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 734/1476] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:51:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:51:07.218321 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c2e44-ae2c-407f-8f39-d859d2adca8c None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:51:07.218796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c2e44-ae2c-407f-8f39-d859d2adca8c None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:51:07.218911 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c2e44-ae2c-407f-8f39-d859d2adca8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:51:07.237516 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-1e4c2e44-ae2c-407f-8f39-d859d2adca8c None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:51:07.238716 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-1e4c2e44-ae2c-407f-8f39-d859d2adca8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:51:07.247517 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 743/1477] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:51:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:51:07.271161 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac75eb4-a053-4506-a5a0-9f228b0154aa None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:51:07.288062 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac75eb4-a053-4506-a5a0-9f228b0154aa None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:51:07.293048 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7059ed5a-846a-49f7-a31a-080b54eb8923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:51:07.293755 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059ed5a-846a-49f7-a31a-080b54eb8923 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:51:07.294035 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059ed5a-846a-49f7-a31a-080b54eb8923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:51:07.294305 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059ed5a-846a-49f7-a31a-080b54eb8923 admin admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:51:07.295047 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7059ed5a-846a-49f7-a31a-080b54eb8923 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:51:07.296865 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: WARNING py.warnings [None req-7059ed5a-846a-49f7-a31a-080b54eb8923 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 01:51:07.296865 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: warnings.warn(msg) Jun 23 01:51:07.296865 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: Jun 23 01:51:07.298029 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7059ed5a-846a-49f7-a31a-080b54eb8923 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:51:07.318526 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 735/1478] 10.222.0.26 () {56 vars in 1108 bytes} [Thu Jun 23 01:51:07 2022] GET /identity/v3/projects => generated 2553 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:51:09.368490 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce14f4c-53a5-4692-b19f-767d25bfefe1 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:51:09.369377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce14f4c-53a5-4692-b19f-767d25bfefe1 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:51:09.369377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce14f4c-53a5-4692-b19f-767d25bfefe1 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:51:09.371850 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 744/1479] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:51:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:51:09.377329 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e11e6-0e5c-4263-90e0-5268fdf43138 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:51:09.377792 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e11e6-0e5c-4263-90e0-5268fdf43138 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:51:09.377896 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e11e6-0e5c-4263-90e0-5268fdf43138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:51:09.403023 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-bf6e11e6-0e5c-4263-90e0-5268fdf43138 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:51:09.404277 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-bf6e11e6-0e5c-4263-90e0-5268fdf43138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:51:09.413413 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 736/1480] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:51:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:51:09.464896 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce14f4c-53a5-4692-b19f-767d25bfefe1 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:51:09.480960 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce14f4c-53a5-4692-b19f-767d25bfefe1 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:51:09.487577 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e794e851-8d90-4e2f-9724-8a9244d63ea0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4e83df33b64a3ebf28f54e2d94fe63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:51:09.488210 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e794e851-8d90-4e2f-9724-8a9244d63ea0 service nova] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:51:09.488486 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e794e851-8d90-4e2f-9724-8a9244d63ea0 service nova] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:51:09.488759 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e794e851-8d90-4e2f-9724-8a9244d63ea0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:51:09.497715 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e794e851-8d90-4e2f-9724-8a9244d63ea0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:51:09.499340 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e794e851-8d90-4e2f-9724-8a9244d63ea0 service nova] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:51:09.508819 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 745/1481] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:51:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:51:11.695266 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1017a-daa7-4a37-b3aa-03364923a217 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:51:11.696134 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1017a-daa7-4a37-b3aa-03364923a217 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:51:11.696134 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1017a-daa7-4a37-b3aa-03364923a217 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:51:11.698384 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 737/1482] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:51:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:51:11.707155 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02567696-b9ed-425b-a467-3f5f43557a9a None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:51:11.707920 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02567696-b9ed-425b-a467-3f5f43557a9a None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:51:11.708043 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02567696-b9ed-425b-a467-3f5f43557a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:51:11.730299 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-02567696-b9ed-425b-a467-3f5f43557a9a None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:51:11.731550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-02567696-b9ed-425b-a467-3f5f43557a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:51:11.740718 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 746/1483] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:51:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:51:11.777105 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1017a-daa7-4a37-b3aa-03364923a217 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:51:11.794524 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1017a-daa7-4a37-b3aa-03364923a217 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:51:11.800322 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b403269e-3b80-4de5-9394-14fd1b3e6e5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:51:11.801300 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b403269e-3b80-4de5-9394-14fd1b3e6e5c service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:51:11.801693 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b403269e-3b80-4de5-9394-14fd1b3e6e5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:51:11.802083 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b403269e-3b80-4de5-9394-14fd1b3e6e5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:51:11.814020 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b403269e-3b80-4de5-9394-14fd1b3e6e5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:51:11.815496 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b403269e-3b80-4de5-9394-14fd1b3e6e5c service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:51:11.824770 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 738/1484] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:51:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:51:13.865068 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e2aea-68a5-44ff-9992-5251feb81cc9 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:51:13.865904 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e2aea-68a5-44ff-9992-5251feb81cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:51:13.865904 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e2aea-68a5-44ff-9992-5251feb81cc9 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:51:13.868056 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 747/1485] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:51:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:51:13.875876 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9a24e-0653-421d-b6fd-3a1bc9d698ab None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:51:13.876305 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9a24e-0653-421d-b6fd-3a1bc9d698ab None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:51:13.876486 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9a24e-0653-421d-b6fd-3a1bc9d698ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:51:13.900591 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-c1d9a24e-0653-421d-b6fd-3a1bc9d698ab None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:51:13.902365 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-c1d9a24e-0653-421d-b6fd-3a1bc9d698ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:51:13.912108 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 739/1486] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:51:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:51:13.953111 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1e2aea-68a5-44ff-9992-5251feb81cc9 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:51:13.970141 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1e2aea-68a5-44ff-9992-5251feb81cc9 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:51:13.974369 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9ffab2-98af-4d6f-a200-fd325038870b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:51:13.975008 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ffab2-98af-4d6f-a200-fd325038870b service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:51:13.975323 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ffab2-98af-4d6f-a200-fd325038870b service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:51:13.975598 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ffab2-98af-4d6f-a200-fd325038870b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:51:13.985108 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9ffab2-98af-4d6f-a200-fd325038870b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:51:13.986550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9ffab2-98af-4d6f-a200-fd325038870b service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:51:13.997091 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 748/1487] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:51:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:51:16.111062 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf006ff-8bb5-49fe-9a01-8a61d6dfa465 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:51:16.111968 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf006ff-8bb5-49fe-9a01-8a61d6dfa465 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:51:16.111968 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf006ff-8bb5-49fe-9a01-8a61d6dfa465 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:51:16.114200 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 740/1488] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:51:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:51:16.120626 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b24d9-00c3-4527-9d0f-321794eb85b0 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:51:16.121369 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b24d9-00c3-4527-9d0f-321794eb85b0 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:51:16.121649 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b24d9-00c3-4527-9d0f-321794eb85b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:51:16.142628 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-3b1b24d9-00c3-4527-9d0f-321794eb85b0 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:51:16.145192 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-3b1b24d9-00c3-4527-9d0f-321794eb85b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:51:16.159374 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 749/1489] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:51:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:51:16.209178 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf006ff-8bb5-49fe-9a01-8a61d6dfa465 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:51:16.221070 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf006ff-8bb5-49fe-9a01-8a61d6dfa465 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:51:16.225518 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a2705c-305f-4aee-9142-d8a8061c3b3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:51:16.226152 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2705c-305f-4aee-9142-d8a8061c3b3b service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:51:16.226432 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2705c-305f-4aee-9142-d8a8061c3b3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:51:16.226705 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2705c-305f-4aee-9142-d8a8061c3b3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:51:16.236645 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a2705c-305f-4aee-9142-d8a8061c3b3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:51:16.238091 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a2705c-305f-4aee-9142-d8a8061c3b3b service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:51:16.247748 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 741/1490] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:51:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:51:18.468426 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d9555-59e2-4e76-9cc4-c9b34ae06301 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:51:18.469013 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d9555-59e2-4e76-9cc4-c9b34ae06301 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:51:18.469013 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d9555-59e2-4e76-9cc4-c9b34ae06301 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:51:18.470357 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 750/1491] 10.222.0.26 () {54 vars in 926 bytes} [Thu Jun 23 01:51:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:51:18.476739 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd41da7e-84c7-4324-8561-b3423563d662 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:51:18.477454 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd41da7e-84c7-4324-8561-b3423563d662 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:51:18.477614 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd41da7e-84c7-4324-8561-b3423563d662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:51:18.502655 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-fd41da7e-84c7-4324-8561-b3423563d662 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:51:18.503886 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-fd41da7e-84c7-4324-8561-b3423563d662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:51:18.512226 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 742/1492] 10.222.0.26 () {58 vars in 1023 bytes} [Thu Jun 23 01:51:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:51:18.554181 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9d9555-59e2-4e76-9cc4-c9b34ae06301 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:51:18.568028 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9d9555-59e2-4e76-9cc4-c9b34ae06301 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:51:18.572181 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8844c-7c16-48c3-86de-d9a84936aff2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:51:18.572816 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8844c-7c16-48c3-86de-d9a84936aff2 service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:51:18.573097 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8844c-7c16-48c3-86de-d9a84936aff2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:51:18.573383 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8844c-7c16-48c3-86de-d9a84936aff2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:51:18.582881 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c8844c-7c16-48c3-86de-d9a84936aff2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:51:18.584353 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c8844c-7c16-48c3-86de-d9a84936aff2 service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:51:18.594026 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 751/1493] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:51:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:51:28.497490 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=33 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:51:28.589704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=33 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:52:50.519309 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1280adb2-07de-4f3b-8acb-73475a872ac1 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:52:50.520227 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1280adb2-07de-4f3b-8acb-73475a872ac1 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:52:50.520227 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1280adb2-07de-4f3b-8acb-73475a872ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:52:50.541389 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-1280adb2-07de-4f3b-8acb-73475a872ac1 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:52:50.542597 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-1280adb2-07de-4f3b-8acb-73475a872ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:52:50.551873 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 743/1494] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:52:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:52:50.563398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c6ea5-5b05-47f1-bef0-3990a144363e None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:52:50.563923 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c6ea5-5b05-47f1-bef0-3990a144363e None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:52:50.563923 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c6ea5-5b05-47f1-bef0-3990a144363e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:52:50.584864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-f06c6ea5-5b05-47f1-bef0-3990a144363e None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:52:50.586202 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-f06c6ea5-5b05-47f1-bef0-3990a144363e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:52:50.595891 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 752/1495] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:52:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 01:52:50.600654 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1280adb2-07de-4f3b-8acb-73475a872ac1 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:52:50.610759 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1280adb2-07de-4f3b-8acb-73475a872ac1 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:52:50.616293 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d65376a-0241-4d41-9c2b-0cb7c276b558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:52:50.616949 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65376a-0241-4d41-9c2b-0cb7c276b558 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:52:50.617222 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65376a-0241-4d41-9c2b-0cb7c276b558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:52:50.617500 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65376a-0241-4d41-9c2b-0cb7c276b558 admin admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:52:50.618235 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d65376a-0241-4d41-9c2b-0cb7c276b558 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:52:50.618539 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d65376a-0241-4d41-9c2b-0cb7c276b558 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:52:50.619643 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: WARNING py.warnings [None req-1d65376a-0241-4d41-9c2b-0cb7c276b558 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 23 01:52:50.619643 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: warnings.warn(msg) Jun 23 01:52:50.619643 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: Jun 23 01:52:50.620523 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d65376a-0241-4d41-9c2b-0cb7c276b558 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:52:50.632397 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 744/1496] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 01:52:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:52:50.640560 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06c6ea5-5b05-47f1-bef0-3990a144363e None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:52:50.650344 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06c6ea5-5b05-47f1-bef0-3990a144363e None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:52:50.656924 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a539892d-5710-43c4-87f4-6a38abfba864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:52:50.657566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539892d-5710-43c4-87f4-6a38abfba864 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:52:50.657849 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539892d-5710-43c4-87f4-6a38abfba864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:52:50.658117 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539892d-5710-43c4-87f4-6a38abfba864 admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:52:50.658828 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a539892d-5710-43c4-87f4-6a38abfba864 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:52:50.660640 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: WARNING py.warnings [None req-a539892d-5710-43c4-87f4-6a38abfba864 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 01:52:50.660640 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: warnings.warn(msg) Jun 23 01:52:50.660640 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: Jun 23 01:52:50.661861 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a539892d-5710-43c4-87f4-6a38abfba864 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:52:50.705159 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 753/1497] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:52:50 2022] POST /identity/v3/projects => generated 379 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:52:50.711617 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d65376a-0241-4d41-9c2b-0cb7c276b558 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:52:50.722457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d65376a-0241-4d41-9c2b-0cb7c276b558 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:52:50.790091 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24312e5-89ef-41f7-9df7-bb2de88eaac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:52:50.790768 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24312e5-89ef-41f7-9df7-bb2de88eaac8 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:52:50.791057 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24312e5-89ef-41f7-9df7-bb2de88eaac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:52:50.791377 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24312e5-89ef-41f7-9df7-bb2de88eaac8 admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:52:50.792072 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24312e5-89ef-41f7-9df7-bb2de88eaac8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:52:50.793095 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: WARNING py.warnings [None req-c24312e5-89ef-41f7-9df7-bb2de88eaac8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 01:52:50.793095 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: warnings.warn(msg) Jun 23 01:52:50.793095 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: Jun 23 01:52:50.794301 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24312e5-89ef-41f7-9df7-bb2de88eaac8 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:52:50.816801 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 745/1498] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:52:50 2022] POST /identity/v3/users => generated 290 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:52:50.821045 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a539892d-5710-43c4-87f4-6a38abfba864 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:52:50.831467 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a539892d-5710-43c4-87f4-6a38abfba864 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:52:50.836216 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80b185-cec0-4436-a8f4-e7f62410f76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:52:50.836861 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80b185-cec0-4436-a8f4-e7f62410f76f admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:52:50.837135 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80b185-cec0-4436-a8f4-e7f62410f76f admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:52:50.837405 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80b185-cec0-4436-a8f4-e7f62410f76f admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:52:50.838214 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b80b185-cec0-4436-a8f4-e7f62410f76f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:52:50.839252 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: WARNING py.warnings [None req-4b80b185-cec0-4436-a8f4-e7f62410f76f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 23 01:52:50.839252 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: warnings.warn(msg) Jun 23 01:52:50.839252 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: Jun 23 01:52:50.840141 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b80b185-cec0-4436-a8f4-e7f62410f76f admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:52:50.852973 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 754/1499] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:52:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:52:50.857136 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24312e5-89ef-41f7-9df7-bb2de88eaac8 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:52:50.867187 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24312e5-89ef-41f7-9df7-bb2de88eaac8 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:52:50.872126 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ddad2-df74-45bc-8307-cad57b7b7613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:52:50.872785 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ddad2-df74-45bc-8307-cad57b7b7613 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:52:50.873075 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ddad2-df74-45bc-8307-cad57b7b7613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:52:50.873355 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ddad2-df74-45bc-8307-cad57b7b7613 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:52:50.874090 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917ddad2-df74-45bc-8307-cad57b7b7613 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:52:50.875115 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: WARNING py.warnings [None req-917ddad2-df74-45bc-8307-cad57b7b7613 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 23 01:52:50.875115 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: warnings.warn(msg) Jun 23 01:52:50.875115 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: Jun 23 01:52:50.876022 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917ddad2-df74-45bc-8307-cad57b7b7613 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:52:50.889703 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 746/1500] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:52:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:52:50.896109 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80b185-cec0-4436-a8f4-e7f62410f76f admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:52:50.906560 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80b185-cec0-4436-a8f4-e7f62410f76f admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:52:50.912109 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d22168-255a-4c74-835b-d3c2bbc22856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:52:50.912975 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d22168-255a-4c74-835b-d3c2bbc22856 admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:52:50.913257 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d22168-255a-4c74-835b-d3c2bbc22856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:52:50.913525 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d22168-255a-4c74-835b-d3c2bbc22856 admin admin] PATH_INFO: `/v3/projects/c2d509f35fcc4f8486d78d0e10235e07/users/d6cbefca3383487ea5e10192c295fe11/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:52:50.929308 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d22168-255a-4c74-835b-d3c2bbc22856 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2d509f35fcc4f8486d78d0e10235e07, user_id=d6cbefca3383487ea5e10192c295fe11, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:52:50.930334 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: WARNING py.warnings [None req-21d22168-255a-4c74-835b-d3c2bbc22856 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 01:52:50.930334 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: warnings.warn(msg) Jun 23 01:52:50.930334 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: Jun 23 01:52:50.932429 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d22168-255a-4c74-835b-d3c2bbc22856 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:52:50.947630 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 755/1501] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:52:50 2022] PUT /identity/v3/projects/c2d509f35fcc4f8486d78d0e10235e07/users/d6cbefca3383487ea5e10192c295fe11/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:52:50.959247 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e6aa8-cd75-40f6-996c-49c9e4ebb339 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:52:50.959575 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e6aa8-cd75-40f6-996c-49c9e4ebb339 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:52:50.959857 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e6aa8-cd75-40f6-996c-49c9e4ebb339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:52:50.987724 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-143e6aa8-cd75-40f6-996c-49c9e4ebb339 None None] MFA Rules not processed for user `d6cbefca3383487ea5e10192c295fe11`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:52:50.988949 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-143e6aa8-cd75-40f6-996c-49c9e4ebb339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:52:51.057286 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 747/1502] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:52:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:52:51.073136 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf0f3d-79a6-4671-be75-a747694ca37f None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:52:51.073467 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf0f3d-79a6-4671-be75-a747694ca37f None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:52:51.073842 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf0f3d-79a6-4671-be75-a747694ca37f None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:52:51.075509 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 756/1503] 10.222.0.26 () {54 vars in 963 bytes} [Thu Jun 23 01:52:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:52:51.080699 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad2a6fb-7835-4dca-8290-f38ed9ddb5a0 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:52:51.080993 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad2a6fb-7835-4dca-8290-f38ed9ddb5a0 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:52:51.081275 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad2a6fb-7835-4dca-8290-f38ed9ddb5a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:52:51.100029 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-3ad2a6fb-7835-4dca-8290-f38ed9ddb5a0 None None] MFA Rules not processed for user `dc5bb02f8eb64ac980e3218470b3c1b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:52:51.101262 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-3ad2a6fb-7835-4dca-8290-f38ed9ddb5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:52:51.145438 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:52:51.155730 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 748/1504] 10.222.0.26 () {58 vars in 1060 bytes} [Thu Jun 23 01:52:51 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 01:52:51.167452 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf0f3d-79a6-4671-be75-a747694ca37f None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:52:51.178893 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf0f3d-79a6-4671-be75-a747694ca37f None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:52:51.182798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842cbc2c-6d95-4b31-9361-2399408460f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:52:51.183446 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842cbc2c-6d95-4b31-9361-2399408460f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:52:51.183724 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842cbc2c-6d95-4b31-9361-2399408460f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:52:51.183995 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842cbc2c-6d95-4b31-9361-2399408460f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:52:51.192236 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842cbc2c-6d95-4b31-9361-2399408460f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:52:51.193665 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842cbc2c-6d95-4b31-9361-2399408460f7 service cinder] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:52:51.202691 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 757/1505] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:52:51 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:52:51.244289 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2811af51-2735-4503-93e6-9c2ddb536009 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:52:51.244608 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2811af51-2735-4503-93e6-9c2ddb536009 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:52:51.244896 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2811af51-2735-4503-93e6-9c2ddb536009 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:52:51.246210 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 749/1506] 10.222.0.26 () {52 vars in 829 bytes} [Thu Jun 23 01:52:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:52:51.250073 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842cbc2c-6d95-4b31-9361-2399408460f7 service cinder] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:52:51.318926 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842cbc2c-6d95-4b31-9361-2399408460f7 service cinder] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:52:51.324243 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be911084-b851-4546-9705-55da0f114bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:52:51.325066 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be911084-b851-4546-9705-55da0f114bbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:52:51.325344 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be911084-b851-4546-9705-55da0f114bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:52:51.325617 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be911084-b851-4546-9705-55da0f114bbe admin admin] PATH_INFO: `/v3/users/d6cbefca3383487ea5e10192c295fe11` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:52:51.326858 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be911084-b851-4546-9705-55da0f114bbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6cbefca3383487ea5e10192c295fe11)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:52:51.327930 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: WARNING py.warnings [None req-be911084-b851-4546-9705-55da0f114bbe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 01:52:51.327930 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: warnings.warn(msg) Jun 23 01:52:51.327930 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: Jun 23 01:52:51.329174 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be911084-b851-4546-9705-55da0f114bbe admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:52:51.406565 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-be911084-b851-4546-9705-55da0f114bbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6cbefca3383487ea5e10192c295fe11'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:52:51.424716 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-be911084-b851-4546-9705-55da0f114bbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6cbefca3383487ea5e10192c295fe11'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:52:51.436553 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-be911084-b851-4546-9705-55da0f114bbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6cbefca3383487ea5e10192c295fe11'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:52:51.445675 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:52:51.451031 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-be911084-b851-4546-9705-55da0f114bbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6cbefca3383487ea5e10192c295fe11'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:52:51.452059 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-be911084-b851-4546-9705-55da0f114bbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6cbefca3383487ea5e10192c295fe11'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:52:51.459199 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 758/1507] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:52:51 2022] DELETE /identity/v3/users/d6cbefca3383487ea5e10192c295fe11 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:52:51.463792 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5e8e9-1d73-4779-ae44-de4c687aeee7 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:52:51.464081 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5e8e9-1d73-4779-ae44-de4c687aeee7 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:52:51.464360 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5e8e9-1d73-4779-ae44-de4c687aeee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:52:51.482203 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-0dd5e8e9-1d73-4779-ae44-de4c687aeee7 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:52:51.483493 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-0dd5e8e9-1d73-4779-ae44-de4c687aeee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:52:51.551704 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 750/1508] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:52:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 01:52:51.561930 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be911084-b851-4546-9705-55da0f114bbe admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:52:51.563468 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-be911084-b851-4546-9705-55da0f114bbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:52:51.627591 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be911084-b851-4546-9705-55da0f114bbe admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:52:51.631700 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae9b368-ebf4-4c9f-8419-bfb0c43c63ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:52:51.632366 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae9b368-ebf4-4c9f-8419-bfb0c43c63ce service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:52:51.632640 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae9b368-ebf4-4c9f-8419-bfb0c43c63ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:52:51.632921 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae9b368-ebf4-4c9f-8419-bfb0c43c63ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:52:51.643735 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae9b368-ebf4-4c9f-8419-bfb0c43c63ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:52:51.645201 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae9b368-ebf4-4c9f-8419-bfb0c43c63ce service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:52:51.655625 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 759/1509] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:52:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:52:52.001843 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd5e8e9-1d73-4779-ae44-de4c687aeee7 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:52:52.003396 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-0dd5e8e9-1d73-4779-ae44-de4c687aeee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:52:52.015957 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd5e8e9-1d73-4779-ae44-de4c687aeee7 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:52:52.020923 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f8cce-6e20-4e88-a749-377d279ffc36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:52:52.022243 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f8cce-6e20-4e88-a749-377d279ffc36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:52:52.022243 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f8cce-6e20-4e88-a749-377d279ffc36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:52:52.022471 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f8cce-6e20-4e88-a749-377d279ffc36 admin admin] PATH_INFO: `/v3/projects/c2d509f35fcc4f8486d78d0e10235e07` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:52:52.023614 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6f8cce-6e20-4e88-a749-377d279ffc36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2d509f35fcc4f8486d78d0e10235e07)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:52:52.024681 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: WARNING py.warnings [None req-8d6f8cce-6e20-4e88-a749-377d279ffc36 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 01:52:52.024681 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: warnings.warn(msg) Jun 23 01:52:52.024681 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: Jun 23 01:52:52.026016 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6f8cce-6e20-4e88-a749-377d279ffc36 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:52:52.091155 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-8d6f8cce-6e20-4e88-a749-377d279ffc36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2d509f35fcc4f8486d78d0e10235e07', 'request_id': 'req-8d6f8cce-6e20-4e88-a749-377d279ffc36'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:52:52.104709 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-8d6f8cce-6e20-4e88-a749-377d279ffc36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2d509f35fcc4f8486d78d0e10235e07', 'request_id': 'req-8d6f8cce-6e20-4e88-a749-377d279ffc36'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:52:52.114022 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-8d6f8cce-6e20-4e88-a749-377d279ffc36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2d509f35fcc4f8486d78d0e10235e07', 'request_id': 'req-8d6f8cce-6e20-4e88-a749-377d279ffc36'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:52:52.125370 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-8d6f8cce-6e20-4e88-a749-377d279ffc36 admin admin] The token cache is being invalidate because project c2d509f35fcc4f8486d78d0e10235e07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:52:52.125657 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-8d6f8cce-6e20-4e88-a749-377d279ffc36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:52:52.126365 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-8d6f8cce-6e20-4e88-a749-377d279ffc36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:52:52.127953 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 751/1510] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:52:52 2022] DELETE /identity/v3/projects/c2d509f35fcc4f8486d78d0e10235e07 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:53:01.650770 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=29 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:53:02.122188 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:54:28.340489 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9376f5-4cd9-4ce3-b2fb-3f4bf49adce2 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:54:28.341074 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9376f5-4cd9-4ce3-b2fb-3f4bf49adce2 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:54:28.341074 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9376f5-4cd9-4ce3-b2fb-3f4bf49adce2 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:54:28.342616 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 760/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 01:54:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:54:28.347670 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c02aa-a027-42c8-a0a8-28fd333077ef None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:54:28.348163 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c02aa-a027-42c8-a0a8-28fd333077ef None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:54:28.348269 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c02aa-a027-42c8-a0a8-28fd333077ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:54:28.367301 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-e68c02aa-a027-42c8-a0a8-28fd333077ef None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:54:28.368534 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-e68c02aa-a027-42c8-a0a8-28fd333077ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:54:28.433265 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 752/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 01:54:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:54:28.590219 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9376f5-4cd9-4ce3-b2fb-3f4bf49adce2 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:54:28.593564 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-5a9376f5-4cd9-4ce3-b2fb-3f4bf49adce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:54:28.683592 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9376f5-4cd9-4ce3-b2fb-3f4bf49adce2 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:54:28.689751 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e476b82-033d-4969-89c1-279304e6d89b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4e83df33b64a3ebf28f54e2d94fe63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:54:28.690393 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e476b82-033d-4969-89c1-279304e6d89b service nova] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:54:28.690669 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e476b82-033d-4969-89c1-279304e6d89b service nova] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:54:28.690939 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e476b82-033d-4969-89c1-279304e6d89b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:54:28.704757 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e476b82-033d-4969-89c1-279304e6d89b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:54:28.707405 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e476b82-033d-4969-89c1-279304e6d89b service nova] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:54:28.719298 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 761/1513] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:54:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:54:29.554561 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb30456-1404-482d-b46a-d485bcb348bd None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:54:29.555167 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb30456-1404-482d-b46a-d485bcb348bd None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:54:29.555167 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb30456-1404-482d-b46a-d485bcb348bd None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:54:29.556567 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 753/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 01:54:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:54:29.562546 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb35bbc-d673-44a5-9c03-3b7b82c674ab None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:54:29.563282 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb35bbc-d673-44a5-9c03-3b7b82c674ab None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:54:29.563489 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb35bbc-d673-44a5-9c03-3b7b82c674ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:54:29.592398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-cdb35bbc-d673-44a5-9c03-3b7b82c674ab None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:54:29.593552 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-cdb35bbc-d673-44a5-9c03-3b7b82c674ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:54:29.603070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 762/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 01:54:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:54:29.753902 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb30456-1404-482d-b46a-d485bcb348bd None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:54:29.771498 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb30456-1404-482d-b46a-d485bcb348bd None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:54:29.777685 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1059304d-2361-433a-915e-123eff880385 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4e83df33b64a3ebf28f54e2d94fe63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:54:29.778329 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059304d-2361-433a-915e-123eff880385 service nova] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:54:29.778607 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059304d-2361-433a-915e-123eff880385 service nova] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:54:29.778877 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059304d-2361-433a-915e-123eff880385 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:54:29.787239 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1059304d-2361-433a-915e-123eff880385 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:54:29.788898 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1059304d-2361-433a-915e-123eff880385 service nova] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:54:29.798457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 754/1516] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:54:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:54:30.113264 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c415c-5853-43fe-9b50-f7e72bfea2e4 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:54:30.113523 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c415c-5853-43fe-9b50-f7e72bfea2e4 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:54:30.113802 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c415c-5853-43fe-9b50-f7e72bfea2e4 None None] PATH_INFO: `/v3` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:54:30.115129 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 763/1517] 10.222.0.16 () {54 vars in 925 bytes} [Thu Jun 23 01:54:30 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:54:30.122121 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d7bb80-b3bf-4948-848e-fd8031ded960 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:54:30.122349 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d7bb80-b3bf-4948-848e-fd8031ded960 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:54:30.122653 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d7bb80-b3bf-4948-848e-fd8031ded960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:54:30.142373 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-97d7bb80-b3bf-4948-848e-fd8031ded960 None None] MFA Rules not processed for user `e5bf3baee1b843fa89b65e5485f6bebb`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:54:30.143629 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-97d7bb80-b3bf-4948-848e-fd8031ded960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:54:30.209591 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 755/1518] 10.222.0.16 () {58 vars in 1015 bytes} [Thu Jun 23 01:54:30 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:54:33.951489 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01cb89-efa2-43a8-9a4e-fbc77c580695 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:54:33.951986 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01cb89-efa2-43a8-9a4e-fbc77c580695 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:54:33.952112 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01cb89-efa2-43a8-9a4e-fbc77c580695 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:54:33.953550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 764/1519] 10.222.0.26 () {54 vars in 960 bytes} [Thu Jun 23 01:54:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:54:33.958785 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98011664-3d77-41d7-bf60-32769d32bbab None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:54:33.959246 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98011664-3d77-41d7-bf60-32769d32bbab None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:54:33.959344 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98011664-3d77-41d7-bf60-32769d32bbab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:54:33.978349 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-98011664-3d77-41d7-bf60-32769d32bbab None None] MFA Rules not processed for user `e5bf3baee1b843fa89b65e5485f6bebb`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:54:33.979782 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-98011664-3d77-41d7-bf60-32769d32bbab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:54:33.990239 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 756/1520] 10.222.0.26 () {58 vars in 1057 bytes} [Thu Jun 23 01:54:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:54:34.001558 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af01cb89-efa2-43a8-9a4e-fbc77c580695 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:54:34.016650 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af01cb89-efa2-43a8-9a4e-fbc77c580695 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:54:34.022321 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb1ca4-fc62-449e-ba4e-1ce83f2c52a3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5bf3baee1b843fa89b65e5485f6bebb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:54:34.023007 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb1ca4-fc62-449e-ba4e-1ce83f2c52a3 service placement] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:54:34.023364 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb1ca4-fc62-449e-ba4e-1ce83f2c52a3 service placement] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:54:34.023643 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb1ca4-fc62-449e-ba4e-1ce83f2c52a3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:54:34.032605 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbb1ca4-fc62-449e-ba4e-1ce83f2c52a3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:54:34.034203 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbb1ca4-fc62-449e-ba4e-1ce83f2c52a3 service placement] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:54:34.044316 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 765/1521] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:54:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 01:54:41.062638 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2060c-e867-4eb0-a732-a17522286b15 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:54:41.063281 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2060c-e867-4eb0-a732-a17522286b15 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:54:41.063281 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2060c-e867-4eb0-a732-a17522286b15 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:54:41.064603 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 757/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 01:54:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:54:41.073209 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae7490e-b29d-49bc-9431-4b2c56c932a0 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:54:41.073921 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae7490e-b29d-49bc-9431-4b2c56c932a0 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:54:41.074075 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae7490e-b29d-49bc-9431-4b2c56c932a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:54:41.096729 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-aae7490e-b29d-49bc-9431-4b2c56c932a0 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:54:41.097984 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-aae7490e-b29d-49bc-9431-4b2c56c932a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:54:41.106469 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 766/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 01:54:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:54:41.271288 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f2060c-e867-4eb0-a732-a17522286b15 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:54:41.292690 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f2060c-e867-4eb0-a732-a17522286b15 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:54:41.298872 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd34c1c-0623-4012-8cb5-16a709f95401 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4e83df33b64a3ebf28f54e2d94fe63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:54:41.299598 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd34c1c-0623-4012-8cb5-16a709f95401 service nova] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:54:41.299877 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd34c1c-0623-4012-8cb5-16a709f95401 service nova] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:54:41.300343 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd34c1c-0623-4012-8cb5-16a709f95401 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:54:41.311764 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd34c1c-0623-4012-8cb5-16a709f95401 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:54:41.313404 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd34c1c-0623-4012-8cb5-16a709f95401 service nova] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:54:41.323686 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 758/1524] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:54:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:54:42.186083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92042e17-5bc6-412b-bb7f-18087fca9c2d None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:54:42.186656 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92042e17-5bc6-412b-bb7f-18087fca9c2d None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:54:42.186656 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92042e17-5bc6-412b-bb7f-18087fca9c2d None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:54:42.188041 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 767/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 01:54:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:54:42.194124 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbee7ac-3c85-4c86-97c3-f28f784931bd None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:54:42.194851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbee7ac-3c85-4c86-97c3-f28f784931bd None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:54:42.195056 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbee7ac-3c85-4c86-97c3-f28f784931bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:54:42.219317 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-dfbee7ac-3c85-4c86-97c3-f28f784931bd None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:54:42.220596 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-dfbee7ac-3c85-4c86-97c3-f28f784931bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:54:42.229134 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 759/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 01:54:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:54:42.383590 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92042e17-5bc6-412b-bb7f-18087fca9c2d None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:54:42.400471 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92042e17-5bc6-412b-bb7f-18087fca9c2d None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:54:42.407146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f889643-0e44-4389-88fc-16296e28f286 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4e83df33b64a3ebf28f54e2d94fe63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:54:42.407808 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f889643-0e44-4389-88fc-16296e28f286 service nova] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:54:42.408087 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f889643-0e44-4389-88fc-16296e28f286 service nova] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:54:42.408356 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f889643-0e44-4389-88fc-16296e28f286 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:54:42.417219 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f889643-0e44-4389-88fc-16296e28f286 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:54:42.418828 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f889643-0e44-4389-88fc-16296e28f286 service nova] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:54:42.428942 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 768/1527] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:54:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:54:46.281048 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5503cb-0294-4fc7-b0ce-ce03b098b12c None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:54:46.281992 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5503cb-0294-4fc7-b0ce-ce03b098b12c None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:54:46.281992 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5503cb-0294-4fc7-b0ce-ce03b098b12c None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:54:46.284122 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 760/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 01:54:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:54:46.290016 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daec332-6457-42ac-9bb6-09c972544e5f None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:54:46.290458 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daec332-6457-42ac-9bb6-09c972544e5f None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:54:46.290555 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daec332-6457-42ac-9bb6-09c972544e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:54:46.309250 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-3daec332-6457-42ac-9bb6-09c972544e5f None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:54:46.310458 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-3daec332-6457-42ac-9bb6-09c972544e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:54:46.318757 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 769/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 01:54:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:54:46.360687 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5503cb-0294-4fc7-b0ce-ce03b098b12c None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:54:46.364085 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-1f5503cb-0294-4fc7-b0ce-ce03b098b12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:54:46.427972 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5503cb-0294-4fc7-b0ce-ce03b098b12c None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:54:46.432086 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a3dd7-ae51-492e-a381-57eb644be887 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:54:46.432723 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a3dd7-ae51-492e-a381-57eb644be887 service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:54:46.433009 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a3dd7-ae51-492e-a381-57eb644be887 service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:54:46.433277 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a3dd7-ae51-492e-a381-57eb644be887 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:54:46.442680 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6a3dd7-ae51-492e-a381-57eb644be887 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:54:46.444155 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6a3dd7-ae51-492e-a381-57eb644be887 service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:54:46.449372 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:54:46.455477 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 761/1530] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:54:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:54:48.471115 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e66b7-fbb3-4455-82d0-9e83f6f99334 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:54:48.472069 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e66b7-fbb3-4455-82d0-9e83f6f99334 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:54:48.472069 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e66b7-fbb3-4455-82d0-9e83f6f99334 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:54:48.474402 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 770/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 01:54:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:54:48.480576 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644eb06-112b-44b9-9f22-bdb3fdaec5b4 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:54:48.481017 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644eb06-112b-44b9-9f22-bdb3fdaec5b4 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:54:48.481126 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644eb06-112b-44b9-9f22-bdb3fdaec5b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:54:48.509442 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-f644eb06-112b-44b9-9f22-bdb3fdaec5b4 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:54:48.510670 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-f644eb06-112b-44b9-9f22-bdb3fdaec5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:54:48.523821 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 762/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 01:54:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:54:48.564980 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7e66b7-fbb3-4455-82d0-9e83f6f99334 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:54:48.576806 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7e66b7-fbb3-4455-82d0-9e83f6f99334 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:54:48.581081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9741d72-a87c-4ec4-bba5-8d65d482d690 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:54:48.581703 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9741d72-a87c-4ec4-bba5-8d65d482d690 service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:54:48.581978 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9741d72-a87c-4ec4-bba5-8d65d482d690 service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:54:48.582247 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9741d72-a87c-4ec4-bba5-8d65d482d690 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:54:48.591843 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9741d72-a87c-4ec4-bba5-8d65d482d690 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:54:48.593267 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9741d72-a87c-4ec4-bba5-8d65d482d690 service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:54:48.602666 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 771/1533] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:54:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 01:54:52.151321 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f545c0-652a-42ee-9c62-fe7a54c9b41a None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:54:52.151889 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f545c0-652a-42ee-9c62-fe7a54c9b41a None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:54:52.151889 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f545c0-652a-42ee-9c62-fe7a54c9b41a None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:54:52.154047 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 763/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 01:54:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:54:52.162869 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c2d12-cfcb-4d8d-ac89-c87cfb9b0c0f None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:54:52.163315 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c2d12-cfcb-4d8d-ac89-c87cfb9b0c0f None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:54:52.163543 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c2d12-cfcb-4d8d-ac89-c87cfb9b0c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:54:52.170589 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=62 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:54:52.184258 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-790c2d12-cfcb-4d8d-ac89-c87cfb9b0c0f None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:54:52.185470 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-790c2d12-cfcb-4d8d-ac89-c87cfb9b0c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:54:52.194086 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 772/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 01:54:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:54:52.497968 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f545c0-652a-42ee-9c62-fe7a54c9b41a None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:54:52.499967 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-67f545c0-652a-42ee-9c62-fe7a54c9b41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:54:52.561415 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f545c0-652a-42ee-9c62-fe7a54c9b41a None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:54:52.568246 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d22e2d-8f0f-4180-9815-2f9a6a287979 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:54:52.569173 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d22e2d-8f0f-4180-9815-2f9a6a287979 service cinder] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:54:52.569551 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d22e2d-8f0f-4180-9815-2f9a6a287979 service cinder] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:54:52.569937 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d22e2d-8f0f-4180-9815-2f9a6a287979 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:54:52.586337 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d22e2d-8f0f-4180-9815-2f9a6a287979 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:54:52.588599 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d22e2d-8f0f-4180-9815-2f9a6a287979 service cinder] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:54:52.604350 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 764/1536] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:54:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:54:54.631133 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e1e1d-fb41-409a-95d5-1435946ca381 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:54:54.632064 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e1e1d-fb41-409a-95d5-1435946ca381 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:54:54.632064 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e1e1d-fb41-409a-95d5-1435946ca381 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:54:54.634169 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 773/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 01:54:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:54:54.642897 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d868f6-3183-4a58-9a44-c37ca8f329cc None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:54:54.643645 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d868f6-3183-4a58-9a44-c37ca8f329cc None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:54:54.643825 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d868f6-3183-4a58-9a44-c37ca8f329cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:54:54.665770 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-77d868f6-3183-4a58-9a44-c37ca8f329cc None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:54:54.666996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-77d868f6-3183-4a58-9a44-c37ca8f329cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:54:54.676190 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 765/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 01:54:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:54:54.979928 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e1e1d-fb41-409a-95d5-1435946ca381 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:54:54.981841 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-6f6e1e1d-fb41-409a-95d5-1435946ca381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:54:54.995984 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e1e1d-fb41-409a-95d5-1435946ca381 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:54:54.999871 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e259296e-d13d-49de-9528-b8572d7f1991 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:54:55.000497 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259296e-d13d-49de-9528-b8572d7f1991 service cinder] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:54:55.000752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259296e-d13d-49de-9528-b8572d7f1991 service cinder] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:54:55.001050 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259296e-d13d-49de-9528-b8572d7f1991 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:54:55.010429 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e259296e-d13d-49de-9528-b8572d7f1991 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:54:55.011908 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e259296e-d13d-49de-9528-b8572d7f1991 service cinder] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:54:55.021321 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 774/1539] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:54:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:55:04.660318 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=32 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:55:05.017311 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=8 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:20.078156 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a293e5-f137-4a62-b03a-050fe58ca7f3 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:20.101436 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a293e5-f137-4a62-b03a-050fe58ca7f3 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:20.101436 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a293e5-f137-4a62-b03a-050fe58ca7f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:20.108178 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-44a293e5-f137-4a62-b03a-050fe58ca7f3 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:20.109872 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-44a293e5-f137-4a62-b03a-050fe58ca7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:20.124495 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 766/1540] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jun 23 01:56:20.136554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062174a-108d-420d-aaf4-b951450f131f None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:20.137166 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062174a-108d-420d-aaf4-b951450f131f None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:20.137166 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062174a-108d-420d-aaf4-b951450f131f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:20.167347 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-a062174a-108d-420d-aaf4-b951450f131f None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:20.168842 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-a062174a-108d-420d-aaf4-b951450f131f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:20.186170 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 775/1541] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:20.190801 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a293e5-f137-4a62-b03a-050fe58ca7f3 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:20.205541 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a293e5-f137-4a62-b03a-050fe58ca7f3 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:20.210663 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e5ebe-6020-43cb-bb90-932823c3d34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:20.211512 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e5ebe-6020-43cb-bb90-932823c3d34b admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:20.211798 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e5ebe-6020-43cb-bb90-932823c3d34b admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:20.212072 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e5ebe-6020-43cb-bb90-932823c3d34b admin admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:20.212808 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9e5ebe-6020-43cb-bb90-932823c3d34b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:56:20.213166 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9e5ebe-6020-43cb-bb90-932823c3d34b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:20.214782 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9e5ebe-6020-43cb-bb90-932823c3d34b admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:20.233536 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 767/1542] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 01:56:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:56:20.241919 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a062174a-108d-420d-aaf4-b951450f131f None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:20.251897 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a062174a-108d-420d-aaf4-b951450f131f None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:20.259803 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261e497f-98f1-447e-bbef-35ff7cca2908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:20.260702 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e497f-98f1-447e-bbef-35ff7cca2908 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:20.261159 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e497f-98f1-447e-bbef-35ff7cca2908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:20.261609 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e497f-98f1-447e-bbef-35ff7cca2908 admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:20.262475 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261e497f-98f1-447e-bbef-35ff7cca2908 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:20.264609 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261e497f-98f1-447e-bbef-35ff7cca2908 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:20.317999 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 776/1543] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:56:20 2022] POST /identity/v3/projects => generated 403 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:56:20.322302 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e5ebe-6020-43cb-bb90-932823c3d34b admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:20.336283 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e5ebe-6020-43cb-bb90-932823c3d34b admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:20.405715 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7689d687-bfdd-4da4-989d-871e518e058d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:20.406602 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689d687-bfdd-4da4-989d-871e518e058d admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:20.406892 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689d687-bfdd-4da4-989d-871e518e058d admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:20.407170 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689d687-bfdd-4da4-989d-871e518e058d admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:20.408233 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7689d687-bfdd-4da4-989d-871e518e058d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:20.409962 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7689d687-bfdd-4da4-989d-871e518e058d admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:20.444161 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 768/1544] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:56:20 2022] POST /identity/v3/users => generated 302 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:56:20.448854 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261e497f-98f1-447e-bbef-35ff7cca2908 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:20.462776 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261e497f-98f1-447e-bbef-35ff7cca2908 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:20.469840 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1aba4d-7d63-4038-8477-670d4a217718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:20.470640 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1aba4d-7d63-4038-8477-670d4a217718 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:20.470927 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1aba4d-7d63-4038-8477-670d4a217718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:20.471237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1aba4d-7d63-4038-8477-670d4a217718 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:20.472112 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1aba4d-7d63-4038-8477-670d4a217718 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:20.473744 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1aba4d-7d63-4038-8477-670d4a217718 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:20.494280 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 777/1545] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:56:20.498061 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7689d687-bfdd-4da4-989d-871e518e058d admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:20.513293 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7689d687-bfdd-4da4-989d-871e518e058d admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:20.518726 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff3472-6c72-4c9c-a9d1-9d85bbb40978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:20.519679 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff3472-6c72-4c9c-a9d1-9d85bbb40978 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:20.519963 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff3472-6c72-4c9c-a9d1-9d85bbb40978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:20.520242 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff3472-6c72-4c9c-a9d1-9d85bbb40978 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:20.520983 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afff3472-6c72-4c9c-a9d1-9d85bbb40978 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:20.522736 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afff3472-6c72-4c9c-a9d1-9d85bbb40978 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:20.539166 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 769/1546] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:56:20.543559 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1aba4d-7d63-4038-8477-670d4a217718 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:20.554491 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1aba4d-7d63-4038-8477-670d4a217718 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:20.564228 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65335174-866f-403b-834b-7ed792dd0dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:20.565157 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65335174-866f-403b-834b-7ed792dd0dbb admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:20.565440 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65335174-866f-403b-834b-7ed792dd0dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:20.565717 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65335174-866f-403b-834b-7ed792dd0dbb admin admin] PATH_INFO: `/v3/projects/5cf82afb2ad54bb49446a1dd59f19c63/users/b31674ccb4a24bb58ef352113d00a64a/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:20.591787 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65335174-866f-403b-834b-7ed792dd0dbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cf82afb2ad54bb49446a1dd59f19c63, user_id=b31674ccb4a24bb58ef352113d00a64a, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:20.594700 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65335174-866f-403b-834b-7ed792dd0dbb admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:20.625980 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 778/1547] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:56:20 2022] PUT /identity/v3/projects/5cf82afb2ad54bb49446a1dd59f19c63/users/b31674ccb4a24bb58ef352113d00a64a/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:56:20.631517 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b19079e-6ace-4935-8d50-c6db647c79cf None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:20.631815 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b19079e-6ace-4935-8d50-c6db647c79cf None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:20.632167 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b19079e-6ace-4935-8d50-c6db647c79cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:20.661773 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-0b19079e-6ace-4935-8d50-c6db647c79cf None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:20.663165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-0b19079e-6ace-4935-8d50-c6db647c79cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:20.755071 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 770/1548] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:20.771236 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65335174-866f-403b-834b-7ed792dd0dbb admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:20.832355 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819ecbf-b04e-4d61-9eca-744932fb71ef None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:20.832737 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819ecbf-b04e-4d61-9eca-744932fb71ef None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:20.832929 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819ecbf-b04e-4d61-9eca-744932fb71ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:20.867849 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65335174-866f-403b-834b-7ed792dd0dbb admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:20.874155 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-4819ecbf-b04e-4d61-9eca-744932fb71ef None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:20.883005 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65775e2f-a406-4a0c-ba09-5fe279fb5f14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:20.883707 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65775e2f-a406-4a0c-ba09-5fe279fb5f14 service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:20.883989 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65775e2f-a406-4a0c-ba09-5fe279fb5f14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:20.884264 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65775e2f-a406-4a0c-ba09-5fe279fb5f14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:20.885476 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-4819ecbf-b04e-4d61-9eca-744932fb71ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:20.907462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 771/1549] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 01:56:20.908600 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f5561-96ce-48d2-86ec-68ba5a8406b7 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:20.908884 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f5561-96ce-48d2-86ec-68ba5a8406b7 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:20.909162 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f5561-96ce-48d2-86ec-68ba5a8406b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:20.916780 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65775e2f-a406-4a0c-ba09-5fe279fb5f14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:20.918535 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65775e2f-a406-4a0c-ba09-5fe279fb5f14 service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:20.933680 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 779/1550] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:56:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:56:20.935028 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a7d7a-fd1a-48a6-85b1-f88edd343656 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:20.935320 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a7d7a-fd1a-48a6-85b1-f88edd343656 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:20.935597 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a7d7a-fd1a-48a6-85b1-f88edd343656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:20.949139 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-e44f5561-96ce-48d2-86ec-68ba5a8406b7 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:20.950825 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-e44f5561-96ce-48d2-86ec-68ba5a8406b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:20.976217 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 772/1551] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:20.977835 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35f1cc-0358-4d59-99b6-052e6a9bf42a None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:20.978131 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35f1cc-0358-4d59-99b6-052e6a9bf42a None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:20.983239 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35f1cc-0358-4d59-99b6-052e6a9bf42a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:20.985548 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-294a7d7a-fd1a-48a6-85b1-f88edd343656 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:20.987014 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-294a7d7a-fd1a-48a6-85b1-f88edd343656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:21.011303 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 780/1552] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:21.017570 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377f355-dfd9-4a64-88fc-82d50f9877e5 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:21.018273 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377f355-dfd9-4a64-88fc-82d50f9877e5 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:21.018772 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377f355-dfd9-4a64-88fc-82d50f9877e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:21.032519 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-9c35f1cc-0358-4d59-99b6-052e6a9bf42a None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:21.033797 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-9c35f1cc-0358-4d59-99b6-052e6a9bf42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:21.049507 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-9377f355-dfd9-4a64-88fc-82d50f9877e5 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:21.051038 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-9377f355-dfd9-4a64-88fc-82d50f9877e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:21.052711 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 773/1553] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:21.054263 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c44b31-5f4f-49f3-ac39-f1143fd41a18 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:21.054555 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c44b31-5f4f-49f3-ac39-f1143fd41a18 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:21.055081 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c44b31-5f4f-49f3-ac39-f1143fd41a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:21.068903 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 781/1554] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:21.069858 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9377f355-dfd9-4a64-88fc-82d50f9877e5 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:21.088455 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9377f355-dfd9-4a64-88fc-82d50f9877e5 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:21.095868 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-57c44b31-5f4f-49f3-ac39-f1143fd41a18 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:21.097186 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8faea2-902e-4e61-b034-e2b56aa649c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:21.097921 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8faea2-902e-4e61-b034-e2b56aa649c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:21.098203 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8faea2-902e-4e61-b034-e2b56aa649c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:21.098527 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8faea2-902e-4e61-b034-e2b56aa649c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:21.098877 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-57c44b31-5f4f-49f3-ac39-f1143fd41a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:21.099497 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8faea2-902e-4e61-b034-e2b56aa649c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:56:21.099809 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8faea2-902e-4e61-b034-e2b56aa649c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:21.100938 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: WARNING py.warnings [None req-5b8faea2-902e-4e61-b034-e2b56aa649c7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 23 01:56:21.100938 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: warnings.warn(msg) Jun 23 01:56:21.100938 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: Jun 23 01:56:21.101939 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8faea2-902e-4e61-b034-e2b56aa649c7 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:21.109204 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 774/1555] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:21.111135 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede94b7-d389-4903-ae22-2ebed67cf44a None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:21.111683 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede94b7-d389-4903-ae22-2ebed67cf44a None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:21.112164 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede94b7-d389-4903-ae22-2ebed67cf44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:21.121290 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 782/1556] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 01:56:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:56:21.129883 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b0dce-7630-40a1-a8ef-82233187ca14 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:21.130203 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b0dce-7630-40a1-a8ef-82233187ca14 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:21.130491 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b0dce-7630-40a1-a8ef-82233187ca14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:21.133851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-0ede94b7-d389-4903-ae22-2ebed67cf44a None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:21.135485 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-0ede94b7-d389-4903-ae22-2ebed67cf44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:21.145002 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 775/1557] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:21.147149 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c63440-951f-45e9-846c-753bb4a485a1 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:21.148029 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c63440-951f-45e9-846c-753bb4a485a1 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:21.148668 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c63440-951f-45e9-846c-753bb4a485a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:21.161376 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-0a5b0dce-7630-40a1-a8ef-82233187ca14 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:21.163175 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-0a5b0dce-7630-40a1-a8ef-82233187ca14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:21.170889 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-98c63440-951f-45e9-846c-753bb4a485a1 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:21.172439 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-98c63440-951f-45e9-846c-753bb4a485a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:21.173956 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 783/1558] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:21.176005 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347cf53-6774-4374-9f50-728af64e9152 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:21.176363 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347cf53-6774-4374-9f50-728af64e9152 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:21.176716 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347cf53-6774-4374-9f50-728af64e9152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:21.185627 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 776/1559] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:21.189337 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c63440-951f-45e9-846c-753bb4a485a1 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:21.200694 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-9347cf53-6774-4374-9f50-728af64e9152 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:21.202060 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-9347cf53-6774-4374-9f50-728af64e9152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:21.202756 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c63440-951f-45e9-846c-753bb4a485a1 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:21.207739 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bfd5a9-cb98-4ca4-a426-d27bb59b03c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:21.208546 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bfd5a9-cb98-4ca4-a426-d27bb59b03c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:21.208829 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bfd5a9-cb98-4ca4-a426-d27bb59b03c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:21.209100 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bfd5a9-cb98-4ca4-a426-d27bb59b03c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:21.209829 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bfd5a9-cb98-4ca4-a426-d27bb59b03c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:56:21.210208 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bfd5a9-cb98-4ca4-a426-d27bb59b03c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:21.211890 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bfd5a9-cb98-4ca4-a426-d27bb59b03c4 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:21.215609 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 784/1560] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:21.222995 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9347cf53-6774-4374-9f50-728af64e9152 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:21.235288 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9347cf53-6774-4374-9f50-728af64e9152 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:21.239720 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 777/1561] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 01:56:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:56:21.242251 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc26d5-e2c5-40a3-9a79-9da79c418316 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:21.242846 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc26d5-e2c5-40a3-9a79-9da79c418316 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:21.242846 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc26d5-e2c5-40a3-9a79-9da79c418316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:21.245193 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ef2fd6-7299-43db-8b9a-3e73f2c2bf17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:21.246028 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef2fd6-7299-43db-8b9a-3e73f2c2bf17 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:21.246591 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef2fd6-7299-43db-8b9a-3e73f2c2bf17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:21.247456 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef2fd6-7299-43db-8b9a-3e73f2c2bf17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:21.248289 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ef2fd6-7299-43db-8b9a-3e73f2c2bf17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:21.250579 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ef2fd6-7299-43db-8b9a-3e73f2c2bf17 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:21.258219 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:21.264249 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-acbc26d5-e2c5-40a3-9a79-9da79c418316 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:21.266238 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-acbc26d5-e2c5-40a3-9a79-9da79c418316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:21.275739 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 778/1562] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:21.276935 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595eeee6-05fa-4c16-9ccf-9c7065522c37 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:21.277291 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595eeee6-05fa-4c16-9ccf-9c7065522c37 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:21.277633 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595eeee6-05fa-4c16-9ccf-9c7065522c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:21.283358 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 785/1563] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:56:21 2022] POST /identity/v3/projects => generated 403 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:56:21.283358 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ef2fd6-7299-43db-8b9a-3e73f2c2bf17 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:21.301464 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-595eeee6-05fa-4c16-9ccf-9c7065522c37 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:21.302739 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-595eeee6-05fa-4c16-9ccf-9c7065522c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:21.333362 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:21.358831 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ef2fd6-7299-43db-8b9a-3e73f2c2bf17 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:21.364169 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1e652-9f26-402c-9edf-64dec3838b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:21.365064 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1e652-9f26-402c-9edf-64dec3838b7b admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:21.365352 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1e652-9f26-402c-9edf-64dec3838b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:21.365727 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1e652-9f26-402c-9edf-64dec3838b7b admin admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:21.366621 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d1e652-9f26-402c-9edf-64dec3838b7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:56:21.367026 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d1e652-9f26-402c-9edf-64dec3838b7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:21.368815 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d1e652-9f26-402c-9edf-64dec3838b7b admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:21.379096 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 779/1564] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:21.381045 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595eeee6-05fa-4c16-9ccf-9c7065522c37 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:21.384668 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 786/1565] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 01:56:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:56:21.391944 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595eeee6-05fa-4c16-9ccf-9c7065522c37 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:21.393165 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f6335-9070-44f7-a17a-2b9d20995e51 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:21.393459 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f6335-9070-44f7-a17a-2b9d20995e51 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:21.393742 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f6335-9070-44f7-a17a-2b9d20995e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:21.396649 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b0fb26-b96d-4b63-97f9-dae53d6366f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:21.397314 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b0fb26-b96d-4b63-97f9-dae53d6366f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:21.397632 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b0fb26-b96d-4b63-97f9-dae53d6366f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:21.397904 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b0fb26-b96d-4b63-97f9-dae53d6366f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:21.398632 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b0fb26-b96d-4b63-97f9-dae53d6366f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:56:21.398935 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b0fb26-b96d-4b63-97f9-dae53d6366f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:21.400938 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b0fb26-b96d-4b63-97f9-dae53d6366f5 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:21.413432 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-d75f6335-9070-44f7-a17a-2b9d20995e51 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:21.414697 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-d75f6335-9070-44f7-a17a-2b9d20995e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:21.421673 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 780/1566] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 01:56:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:56:21.423303 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b0fb26-b96d-4b63-97f9-dae53d6366f5 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:21.423455 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 787/1567] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:21.424766 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98022385-76af-4e4c-b02f-d0d8686cc081 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:21.425389 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98022385-76af-4e4c-b02f-d0d8686cc081 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:21.425652 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98022385-76af-4e4c-b02f-d0d8686cc081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:21.438564 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b0fb26-b96d-4b63-97f9-dae53d6366f5 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:21.443453 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d732fa45-abb1-4e76-a656-a0ed447bd996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:21.444138 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732fa45-abb1-4e76-a656-a0ed447bd996 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:21.444488 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732fa45-abb1-4e76-a656-a0ed447bd996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:21.444772 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732fa45-abb1-4e76-a656-a0ed447bd996 admin admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:21.445478 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d732fa45-abb1-4e76-a656-a0ed447bd996 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:21.446512 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: WARNING py.warnings [None req-d732fa45-abb1-4e76-a656-a0ed447bd996 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 01:56:21.446512 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: warnings.warn(msg) Jun 23 01:56:21.446512 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: Jun 23 01:56:21.447881 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d732fa45-abb1-4e76-a656-a0ed447bd996 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:21.454603 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-98022385-76af-4e4c-b02f-d0d8686cc081 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:21.456121 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-98022385-76af-4e4c-b02f-d0d8686cc081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:21.471168 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 788/1568] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:21.475351 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98022385-76af-4e4c-b02f-d0d8686cc081 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:21.486832 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 781/1569] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:56:21 2022] POST /identity/v3/projects => generated 393 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:56:21.486832 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d732fa45-abb1-4e76-a656-a0ed447bd996 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:21.487139 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98022385-76af-4e4c-b02f-d0d8686cc081 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:21.555771 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d732fa45-abb1-4e76-a656-a0ed447bd996 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:21.556045 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72915797-2735-477b-aeba-b3679081e5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:21.557048 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72915797-2735-477b-aeba-b3679081e5ba admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:21.557326 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72915797-2735-477b-aeba-b3679081e5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:21.557719 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72915797-2735-477b-aeba-b3679081e5ba admin admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:21.558496 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72915797-2735-477b-aeba-b3679081e5ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:21.559603 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: WARNING py.warnings [None req-72915797-2735-477b-aeba-b3679081e5ba admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 01:56:21.559603 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: warnings.warn(msg) Jun 23 01:56:21.559603 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: Jun 23 01:56:21.560908 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72915797-2735-477b-aeba-b3679081e5ba admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:21.561508 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf571a5-24d5-4538-b4de-e9d8ebbe50cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:21.562272 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf571a5-24d5-4538-b4de-e9d8ebbe50cd admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:21.562653 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf571a5-24d5-4538-b4de-e9d8ebbe50cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:21.563033 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf571a5-24d5-4538-b4de-e9d8ebbe50cd admin admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:21.563910 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf571a5-24d5-4538-b4de-e9d8ebbe50cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:56:21.564318 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf571a5-24d5-4538-b4de-e9d8ebbe50cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:21.566137 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf571a5-24d5-4538-b4de-e9d8ebbe50cd admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:21.580402 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 782/1570] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 01:56:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:56:21.581388 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf571a5-24d5-4538-b4de-e9d8ebbe50cd admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:21.591919 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 789/1571] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:56:21 2022] POST /identity/v3/users => generated 302 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:56:21.593024 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72915797-2735-477b-aeba-b3679081e5ba admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:21.593167 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf571a5-24d5-4538-b4de-e9d8ebbe50cd admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:21.598823 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86233ce7-dec0-4252-90dc-a076cf0e1ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:21.599723 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86233ce7-dec0-4252-90dc-a076cf0e1ffd admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:21.600023 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86233ce7-dec0-4252-90dc-a076cf0e1ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:21.600383 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86233ce7-dec0-4252-90dc-a076cf0e1ffd admin admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:21.601134 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86233ce7-dec0-4252-90dc-a076cf0e1ffd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:21.603628 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86233ce7-dec0-4252-90dc-a076cf0e1ffd admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:21.603896 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72915797-2735-477b-aeba-b3679081e5ba admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:21.609277 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662a5751-5298-45f2-949e-4f5cdc4a7fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:21.609957 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a5751-5298-45f2-949e-4f5cdc4a7fa8 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:21.610227 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a5751-5298-45f2-949e-4f5cdc4a7fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:21.610502 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a5751-5298-45f2-949e-4f5cdc4a7fa8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:21.611250 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662a5751-5298-45f2-949e-4f5cdc4a7fa8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:56:21.611561 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662a5751-5298-45f2-949e-4f5cdc4a7fa8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:21.613147 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662a5751-5298-45f2-949e-4f5cdc4a7fa8 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:21.633635 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 790/1572] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 01:56:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:56:21.633635 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662a5751-5298-45f2-949e-4f5cdc4a7fa8 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:21.639512 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662a5751-5298-45f2-949e-4f5cdc4a7fa8 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:21.639740 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 783/1573] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:56:21 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:56:21.640722 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86233ce7-dec0-4252-90dc-a076cf0e1ffd admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:21.691739 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:21.711299 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db8f63-ae47-4f77-aba2-cdb02103704d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:21.712030 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db8f63-ae47-4f77-aba2-cdb02103704d admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:21.712316 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db8f63-ae47-4f77-aba2-cdb02103704d admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:21.712961 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db8f63-ae47-4f77-aba2-cdb02103704d admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:21.713338 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73db8f63-ae47-4f77-aba2-cdb02103704d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:21.715270 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73db8f63-ae47-4f77-aba2-cdb02103704d admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:21.716370 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86233ce7-dec0-4252-90dc-a076cf0e1ffd admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:21.721720 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4106d09-4b38-4d8b-bc38-69aa9df6504a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:21.722835 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4106d09-4b38-4d8b-bc38-69aa9df6504a admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:21.722835 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4106d09-4b38-4d8b-bc38-69aa9df6504a admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:21.729749 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4106d09-4b38-4d8b-bc38-69aa9df6504a admin admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:21.731179 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4106d09-4b38-4d8b-bc38-69aa9df6504a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:56:21.731529 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4106d09-4b38-4d8b-bc38-69aa9df6504a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:21.733809 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4106d09-4b38-4d8b-bc38-69aa9df6504a admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:21.747988 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 784/1574] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 01:56:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:56:21.748437 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 791/1575] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:56:21 2022] POST /identity/v3/projects => generated 387 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:56:21.748924 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4106d09-4b38-4d8b-bc38-69aa9df6504a admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:21.749768 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db8f63-ae47-4f77-aba2-cdb02103704d admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:21.762289 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4106d09-4b38-4d8b-bc38-69aa9df6504a admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:21.781882 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:21.822755 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c045ec0-6c5b-4cd7-b009-c4941e06c4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:21.823639 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c045ec0-6c5b-4cd7-b009-c4941e06c4e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:21.823923 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c045ec0-6c5b-4cd7-b009-c4941e06c4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:21.824203 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c045ec0-6c5b-4cd7-b009-c4941e06c4e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:21.825042 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c045ec0-6c5b-4cd7-b009-c4941e06c4e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:21.827015 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c045ec0-6c5b-4cd7-b009-c4941e06c4e0 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:21.830777 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db8f63-ae47-4f77-aba2-cdb02103704d admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:21.836113 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be05124a-54ae-45d5-947d-ed8a8ab2f5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:21.836726 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05124a-54ae-45d5-947d-ed8a8ab2f5b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:21.836990 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05124a-54ae-45d5-947d-ed8a8ab2f5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:21.837261 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05124a-54ae-45d5-947d-ed8a8ab2f5b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:21.838083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be05124a-54ae-45d5-947d-ed8a8ab2f5b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:21.840164 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be05124a-54ae-45d5-947d-ed8a8ab2f5b1 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:21.855266 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 785/1576] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:56:21 2022] POST /identity/v3/users => generated 297 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:56:21.855266 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c045ec0-6c5b-4cd7-b009-c4941e06c4e0 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:21.864381 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c045ec0-6c5b-4cd7-b009-c4941e06c4e0 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:21.870223 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d892a-1d81-4e73-8ca9-a82e644f8780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:21.870904 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d892a-1d81-4e73-8ca9-a82e644f8780 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:21.871188 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d892a-1d81-4e73-8ca9-a82e644f8780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:21.871497 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d892a-1d81-4e73-8ca9-a82e644f8780 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:21.872531 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560d892a-1d81-4e73-8ca9-a82e644f8780 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:21.874136 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560d892a-1d81-4e73-8ca9-a82e644f8780 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:21.877460 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 792/1577] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:56:21 2022] POST /identity/v3/projects => generated 367 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:56:21.878500 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be05124a-54ae-45d5-947d-ed8a8ab2f5b1 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:21.886733 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 786/1578] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:56:21.887571 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d892a-1d81-4e73-8ca9-a82e644f8780 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:21.897264 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d892a-1d81-4e73-8ca9-a82e644f8780 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:21.951414 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be05124a-54ae-45d5-947d-ed8a8ab2f5b1 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:21.956073 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b08d60-e14c-4b02-98ce-358d1bc2c822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:21.956281 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c64188-044e-46ad-b3d1-3e07d07a0e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:21.956729 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b08d60-e14c-4b02-98ce-358d1bc2c822 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:21.957013 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c64188-044e-46ad-b3d1-3e07d07a0e66 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:21.957302 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c64188-044e-46ad-b3d1-3e07d07a0e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:21.957580 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c64188-044e-46ad-b3d1-3e07d07a0e66 admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:21.957681 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b08d60-e14c-4b02-98ce-358d1bc2c822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:21.957913 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b08d60-e14c-4b02-98ce-358d1bc2c822 admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:21.958277 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c64188-044e-46ad-b3d1-3e07d07a0e66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:21.958630 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b08d60-e14c-4b02-98ce-358d1bc2c822 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:21.960321 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c64188-044e-46ad-b3d1-3e07d07a0e66 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:21.960740 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b08d60-e14c-4b02-98ce-358d1bc2c822 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:21.984451 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 787/1579] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:56:21 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:56:21.984451 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63339900-089e-4f4e-958b-49c3c8b6f8d0 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:21.984451 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63339900-089e-4f4e-958b-49c3c8b6f8d0 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:21.984451 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63339900-089e-4f4e-958b-49c3c8b6f8d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:21.986887 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 793/1580] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:56:21 2022] POST /identity/v3/projects => generated 379 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:56:21.987719 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b08d60-e14c-4b02-98ce-358d1bc2c822 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:22.011146 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-63339900-089e-4f4e-958b-49c3c8b6f8d0 None None] MFA Rules not processed for user `b31674ccb4a24bb58ef352113d00a64a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:22.012453 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-63339900-089e-4f4e-958b-49c3c8b6f8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:22.041681 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:22.052826 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b08d60-e14c-4b02-98ce-358d1bc2c822 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:22.058022 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6513420e-041b-44cc-a83f-c2d30540c083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:22.058694 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6513420e-041b-44cc-a83f-c2d30540c083 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:22.058977 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6513420e-041b-44cc-a83f-c2d30540c083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:22.059284 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6513420e-041b-44cc-a83f-c2d30540c083 admin admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:22.059979 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6513420e-041b-44cc-a83f-c2d30540c083 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:22.061939 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6513420e-041b-44cc-a83f-c2d30540c083 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:22.078637 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 788/1581] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:21 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:22.081570 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63339900-089e-4f4e-958b-49c3c8b6f8d0 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:22.082974 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 794/1582] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:56:21 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:56:22.083795 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6513420e-041b-44cc-a83f-c2d30540c083 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:22.093150 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63339900-089e-4f4e-958b-49c3c8b6f8d0 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:22.093507 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6513420e-041b-44cc-a83f-c2d30540c083 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:22.097862 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f9aff-b8ee-45d1-8f90-3fb7b4b23f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:22.098343 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9f5f9-d59f-4b7e-a8f6-ae5c818a4580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:22.098533 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f9aff-b8ee-45d1-8f90-3fb7b4b23f85 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:22.098673 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9f5f9-d59f-4b7e-a8f6-ae5c818a4580 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:22.098810 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f9aff-b8ee-45d1-8f90-3fb7b4b23f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:22.098924 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9f5f9-d59f-4b7e-a8f6-ae5c818a4580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:22.099079 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f9aff-b8ee-45d1-8f90-3fb7b4b23f85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:22.099176 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9f5f9-d59f-4b7e-a8f6-ae5c818a4580 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:22.099822 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91f9aff-b8ee-45d1-8f90-3fb7b4b23f85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:22.099929 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da9f5f9-d59f-4b7e-a8f6-ae5c818a4580 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:22.101562 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da9f5f9-d59f-4b7e-a8f6-ae5c818a4580 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:22.101851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91f9aff-b8ee-45d1-8f90-3fb7b4b23f85 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:22.112312 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 795/1583] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:56:22.113076 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9f5f9-d59f-4b7e-a8f6-ae5c818a4580 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:22.124151 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9f5f9-d59f-4b7e-a8f6-ae5c818a4580 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:22.127423 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 789/1584] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:56:22 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:56:22.128232 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f9aff-b8ee-45d1-8f90-3fb7b4b23f85 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:22.131382 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2bc6b1-d07c-44d3-8c2d-3926e8df4e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:22.132491 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2bc6b1-d07c-44d3-8c2d-3926e8df4e21 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:22.132775 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2bc6b1-d07c-44d3-8c2d-3926e8df4e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:22.133055 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2bc6b1-d07c-44d3-8c2d-3926e8df4e21 admin admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:22.133744 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2bc6b1-d07c-44d3-8c2d-3926e8df4e21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:22.135723 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2bc6b1-d07c-44d3-8c2d-3926e8df4e21 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:22.143626 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:22.157648 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 796/1585] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:56:22 2022] POST /identity/v3/users => generated 284 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:56:22.158549 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2bc6b1-d07c-44d3-8c2d-3926e8df4e21 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:22.168003 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2bc6b1-d07c-44d3-8c2d-3926e8df4e21 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:22.192182 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f9aff-b8ee-45d1-8f90-3fb7b4b23f85 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:22.196805 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1f21b6-2bd9-4a4c-a8e7-6a95b72ebdae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:22.197474 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f21b6-2bd9-4a4c-a8e7-6a95b72ebdae admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:22.197755 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f21b6-2bd9-4a4c-a8e7-6a95b72ebdae admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:22.198035 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f21b6-2bd9-4a4c-a8e7-6a95b72ebdae admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:22.198767 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1f21b6-2bd9-4a4c-a8e7-6a95b72ebdae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:22.200522 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1f21b6-2bd9-4a4c-a8e7-6a95b72ebdae admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:22.211177 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 790/1586] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:56:22.212072 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1f21b6-2bd9-4a4c-a8e7-6a95b72ebdae admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:22.221754 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1f21b6-2bd9-4a4c-a8e7-6a95b72ebdae admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:22.226320 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9dde21-315d-451a-977b-e096b02962c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:22.226975 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9dde21-315d-451a-977b-e096b02962c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:22.227282 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9dde21-315d-451a-977b-e096b02962c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:22.227570 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9dde21-315d-451a-977b-e096b02962c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:22.227687 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7044d563-48d4-4518-8583-8db8e2afc93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:22.228254 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9dde21-315d-451a-977b-e096b02962c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:22.228359 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7044d563-48d4-4518-8583-8db8e2afc93b admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:22.228621 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7044d563-48d4-4518-8583-8db8e2afc93b admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:22.228894 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7044d563-48d4-4518-8583-8db8e2afc93b admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:22.229628 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7044d563-48d4-4518-8583-8db8e2afc93b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:22.230178 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9dde21-315d-451a-977b-e096b02962c2 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:22.231300 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7044d563-48d4-4518-8583-8db8e2afc93b admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:22.241428 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 797/1587] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:56:22.242299 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7044d563-48d4-4518-8583-8db8e2afc93b admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:22.251890 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7044d563-48d4-4518-8583-8db8e2afc93b admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:22.253344 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 791/1588] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:56:22 2022] POST /identity/v3/users => generated 290 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:56:22.254930 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9dde21-315d-451a-977b-e096b02962c2 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:22.260115 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3647111f-ed3d-4155-aad5-51d1827cf2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:22.261180 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647111f-ed3d-4155-aad5-51d1827cf2b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:22.261625 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647111f-ed3d-4155-aad5-51d1827cf2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:22.261902 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647111f-ed3d-4155-aad5-51d1827cf2b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:22.263076 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3647111f-ed3d-4155-aad5-51d1827cf2b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:22.265799 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3647111f-ed3d-4155-aad5-51d1827cf2b6 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:22.270826 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9dde21-315d-451a-977b-e096b02962c2 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:22.275735 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620f2f1-47db-4f9a-ad3c-5efbbdddd100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:22.276393 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620f2f1-47db-4f9a-ad3c-5efbbdddd100 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:22.276676 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620f2f1-47db-4f9a-ad3c-5efbbdddd100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:22.276955 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620f2f1-47db-4f9a-ad3c-5efbbdddd100 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:22.277695 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620f2f1-47db-4f9a-ad3c-5efbbdddd100 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:22.279371 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620f2f1-47db-4f9a-ad3c-5efbbdddd100 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:22.291318 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 792/1589] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:56:22.291645 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620f2f1-47db-4f9a-ad3c-5efbbdddd100 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:22.298751 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 798/1590] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:56:22 2022] POST /identity/v3/projects => generated 405 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:56:22.298938 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3647111f-ed3d-4155-aad5-51d1827cf2b6 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:22.302546 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620f2f1-47db-4f9a-ad3c-5efbbdddd100 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:22.360047 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0dc65c-6964-416e-830d-b58059c97e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:22.360752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0dc65c-6964-416e-830d-b58059c97e8f admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:22.361085 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0dc65c-6964-416e-830d-b58059c97e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:22.361368 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0dc65c-6964-416e-830d-b58059c97e8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:22.362132 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0dc65c-6964-416e-830d-b58059c97e8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:22.363771 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0dc65c-6964-416e-830d-b58059c97e8f admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:22.363927 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3647111f-ed3d-4155-aad5-51d1827cf2b6 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:22.368642 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f259f84-6a8e-4c96-909f-1ab8f2230efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:22.369286 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f259f84-6a8e-4c96-909f-1ab8f2230efc admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:22.369578 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f259f84-6a8e-4c96-909f-1ab8f2230efc admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:22.369831 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f259f84-6a8e-4c96-909f-1ab8f2230efc admin admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:22.370545 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f259f84-6a8e-4c96-909f-1ab8f2230efc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:22.372485 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f259f84-6a8e-4c96-909f-1ab8f2230efc admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:22.373842 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 793/1591] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:56:22.373842 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0dc65c-6964-416e-830d-b58059c97e8f admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:22.385237 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0dc65c-6964-416e-830d-b58059c97e8f admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:22.390954 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b293a4-8471-40ec-98e6-57084ae11a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:22.391653 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b293a4-8471-40ec-98e6-57084ae11a49 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:22.391937 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b293a4-8471-40ec-98e6-57084ae11a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:22.392216 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b293a4-8471-40ec-98e6-57084ae11a49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:22.392939 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b293a4-8471-40ec-98e6-57084ae11a49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:22.396478 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 799/1592] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:56:22 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:56:22.396478 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f259f84-6a8e-4c96-909f-1ab8f2230efc admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:22.396717 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b293a4-8471-40ec-98e6-57084ae11a49 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:22.400433 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:22.403571 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f259f84-6a8e-4c96-909f-1ab8f2230efc admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:22.406317 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 794/1593] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:56:22.408778 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12397a15-be75-4730-9630-9a63b3d7014e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:22.409995 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12397a15-be75-4730-9630-9a63b3d7014e admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:22.410292 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b293a4-8471-40ec-98e6-57084ae11a49 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:22.410408 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12397a15-be75-4730-9630-9a63b3d7014e admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:22.410671 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12397a15-be75-4730-9630-9a63b3d7014e admin admin] PATH_INFO: `/v3/projects/e4537bc7993743cabfa3752ab1489fed/users/11551dbcae304eadb0c2252a300346b3/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:22.420985 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b293a4-8471-40ec-98e6-57084ae11a49 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:22.426280 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d671d389-9e32-4df6-956a-cfb7e0ca75f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:22.426940 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d671d389-9e32-4df6-956a-cfb7e0ca75f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:22.427250 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d671d389-9e32-4df6-956a-cfb7e0ca75f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:22.427353 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12397a15-be75-4730-9630-9a63b3d7014e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4537bc7993743cabfa3752ab1489fed, user_id=11551dbcae304eadb0c2252a300346b3, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:22.427536 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d671d389-9e32-4df6-956a-cfb7e0ca75f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:22.428311 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d671d389-9e32-4df6-956a-cfb7e0ca75f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:22.430008 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d671d389-9e32-4df6-956a-cfb7e0ca75f7 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:22.430279 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12397a15-be75-4730-9630-9a63b3d7014e admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:22.440485 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 795/1594] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:56:22.443924 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 800/1595] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:56:22 2022] PUT /identity/v3/projects/e4537bc7993743cabfa3752ab1489fed/users/11551dbcae304eadb0c2252a300346b3/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:56:22.444851 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12397a15-be75-4730-9630-9a63b3d7014e admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:22.447604 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d671d389-9e32-4df6-956a-cfb7e0ca75f7 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:22.457716 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d671d389-9e32-4df6-956a-cfb7e0ca75f7 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:22.482103 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:22.514646 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12397a15-be75-4730-9630-9a63b3d7014e admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:22.520207 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb2e4d-7c64-437a-958d-0999e5006b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:22.520876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb2e4d-7c64-437a-958d-0999e5006b6b admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:22.521160 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb2e4d-7c64-437a-958d-0999e5006b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:22.521435 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb2e4d-7c64-437a-958d-0999e5006b6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:22.521923 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da08f0d6-ee13-4420-b51c-6c83a11c7a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:22.522168 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afb2e4d-7c64-437a-958d-0999e5006b6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:22.522663 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08f0d6-ee13-4420-b51c-6c83a11c7a0e admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:22.522944 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08f0d6-ee13-4420-b51c-6c83a11c7a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:22.523233 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08f0d6-ee13-4420-b51c-6c83a11c7a0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:22.523957 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afb2e4d-7c64-437a-958d-0999e5006b6b admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:22.524063 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da08f0d6-ee13-4420-b51c-6c83a11c7a0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:22.525586 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da08f0d6-ee13-4420-b51c-6c83a11c7a0e admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:22.534140 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 801/1596] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:56:22.534907 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb2e4d-7c64-437a-958d-0999e5006b6b admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:22.535609 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 796/1597] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:56:22.536265 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da08f0d6-ee13-4420-b51c-6c83a11c7a0e admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:22.545725 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da08f0d6-ee13-4420-b51c-6c83a11c7a0e admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:22.549158 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb2e4d-7c64-437a-958d-0999e5006b6b admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:22.550559 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6411c3-12ee-49ab-b9a0-0edd109f6926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:22.551476 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6411c3-12ee-49ab-b9a0-0edd109f6926 admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:22.551758 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6411c3-12ee-49ab-b9a0-0edd109f6926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:22.552034 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6411c3-12ee-49ab-b9a0-0edd109f6926 admin admin] PATH_INFO: `/v3/projects/e7608bb4bbf1435289dc08731d51df22/users/0f92d1dfc6c74b51aec7212f9a244a0a/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:22.554078 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f3f39-ed06-4e10-901b-6bbb3abd9724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:22.554734 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f3f39-ed06-4e10-901b-6bbb3abd9724 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:22.555014 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f3f39-ed06-4e10-901b-6bbb3abd9724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:22.555332 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f3f39-ed06-4e10-901b-6bbb3abd9724 admin admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:22.556021 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412f3f39-ed06-4e10-901b-6bbb3abd9724 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:22.557934 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412f3f39-ed06-4e10-901b-6bbb3abd9724 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:22.567281 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6411c3-12ee-49ab-b9a0-0edd109f6926 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7608bb4bbf1435289dc08731d51df22, user_id=0f92d1dfc6c74b51aec7212f9a244a0a, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:22.568324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: WARNING py.warnings [None req-fb6411c3-12ee-49ab-b9a0-0edd109f6926 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 01:56:22.568324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: warnings.warn(msg) Jun 23 01:56:22.568324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: Jun 23 01:56:22.570352 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6411c3-12ee-49ab-b9a0-0edd109f6926 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:22.578743 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 802/1598] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:56:22 2022] POST /identity/v3/users => generated 309 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:56:22.579705 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f3f39-ed06-4e10-901b-6bbb3abd9724 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:22.584643 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 797/1599] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:56:22 2022] PUT /identity/v3/projects/e7608bb4bbf1435289dc08731d51df22/users/0f92d1dfc6c74b51aec7212f9a244a0a/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:56:22.585817 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6411c3-12ee-49ab-b9a0-0edd109f6926 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:22.592306 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f3f39-ed06-4e10-901b-6bbb3abd9724 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:22.646163 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6411c3-12ee-49ab-b9a0-0edd109f6926 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:22.648335 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e05db2-cf39-4ffb-91f9-f2c2fe03360c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:22.648952 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e05db2-cf39-4ffb-91f9-f2c2fe03360c admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:22.649234 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e05db2-cf39-4ffb-91f9-f2c2fe03360c admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:22.649509 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e05db2-cf39-4ffb-91f9-f2c2fe03360c admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:22.650245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e05db2-cf39-4ffb-91f9-f2c2fe03360c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:22.651141 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134d85f2-0ec9-4465-96ec-18e75eda8ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:22.651833 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d85f2-0ec9-4465-96ec-18e75eda8ea4 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:22.651930 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e05db2-cf39-4ffb-91f9-f2c2fe03360c admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:22.652117 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d85f2-0ec9-4465-96ec-18e75eda8ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:22.652396 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d85f2-0ec9-4465-96ec-18e75eda8ea4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:22.653121 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134d85f2-0ec9-4465-96ec-18e75eda8ea4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:22.654703 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134d85f2-0ec9-4465-96ec-18e75eda8ea4 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:22.661918 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 803/1600] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:56:22.662769 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e05db2-cf39-4ffb-91f9-f2c2fe03360c admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:22.665339 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 798/1601] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:56:22.666590 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa95e9-c043-4ecf-b0ff-8d65b03e54ca None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:22.666886 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa95e9-c043-4ecf-b0ff-8d65b03e54ca None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:22.667156 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aa95e9-c043-4ecf-b0ff-8d65b03e54ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:22.672773 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e05db2-cf39-4ffb-91f9-f2c2fe03360c admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:22.677814 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42830d30-9dfd-4f3c-947f-ed97451d1339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:22.678741 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42830d30-9dfd-4f3c-947f-ed97451d1339 admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:22.679027 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42830d30-9dfd-4f3c-947f-ed97451d1339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:22.679334 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42830d30-9dfd-4f3c-947f-ed97451d1339 admin admin] PATH_INFO: `/v3/projects/f87ecb9409d54151a1265e99e0cf6f0b/users/62bd2ed5d8f34a429d1a78d4498647a4/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:22.693025 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-68aa95e9-c043-4ecf-b0ff-8d65b03e54ca None None] MFA Rules not processed for user `11551dbcae304eadb0c2252a300346b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:22.694116 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42830d30-9dfd-4f3c-947f-ed97451d1339 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f87ecb9409d54151a1265e99e0cf6f0b, user_id=62bd2ed5d8f34a429d1a78d4498647a4, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:22.694280 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-68aa95e9-c043-4ecf-b0ff-8d65b03e54ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:22.696863 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42830d30-9dfd-4f3c-947f-ed97451d1339 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:22.710127 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 804/1602] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:56:22 2022] PUT /identity/v3/projects/f87ecb9409d54151a1265e99e0cf6f0b/users/62bd2ed5d8f34a429d1a78d4498647a4/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:56:22.710986 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42830d30-9dfd-4f3c-947f-ed97451d1339 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:22.747512 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:22.769466 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 799/1603] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:22 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:22.770290 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa95e9-c043-4ecf-b0ff-8d65b03e54ca None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:22.777939 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42830d30-9dfd-4f3c-947f-ed97451d1339 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:22.780143 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aa95e9-c043-4ecf-b0ff-8d65b03e54ca None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:22.782823 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afbd164-a4d9-4361-9082-497bf2d53371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:22.783744 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afbd164-a4d9-4361-9082-497bf2d53371 admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:22.784020 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afbd164-a4d9-4361-9082-497bf2d53371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:22.784292 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afbd164-a4d9-4361-9082-497bf2d53371 admin admin] PATH_INFO: `/v3/projects/d0648b2071364ae09ba6d7a1d45c762c/users/0287bd1260d24e5a94c00367b23c7a84/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:22.784876 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e9f78-e1c6-4e87-97a2-991252098f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:22.785533 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e9f78-e1c6-4e87-97a2-991252098f12 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:22.785813 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e9f78-e1c6-4e87-97a2-991252098f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:22.786084 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e9f78-e1c6-4e87-97a2-991252098f12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:22.786839 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687e9f78-e1c6-4e87-97a2-991252098f12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:22.788566 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687e9f78-e1c6-4e87-97a2-991252098f12 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:22.798454 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 800/1604] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:56:22.799171 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afbd164-a4d9-4361-9082-497bf2d53371 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0648b2071364ae09ba6d7a1d45c762c, user_id=0287bd1260d24e5a94c00367b23c7a84, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:22.799357 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e9f78-e1c6-4e87-97a2-991252098f12 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:22.802136 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afbd164-a4d9-4361-9082-497bf2d53371 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:22.809311 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e9f78-e1c6-4e87-97a2-991252098f12 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:22.815916 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 805/1605] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:56:22 2022] PUT /identity/v3/projects/d0648b2071364ae09ba6d7a1d45c762c/users/0287bd1260d24e5a94c00367b23c7a84/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:56:22.820881 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87b6a8-f49f-499e-853a-0a0bc2a237e8 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:22.821172 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87b6a8-f49f-499e-853a-0a0bc2a237e8 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:22.821454 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87b6a8-f49f-499e-853a-0a0bc2a237e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:22.827785 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:22.840048 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-9f87b6a8-f49f-499e-853a-0a0bc2a237e8 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:22.841282 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-9f87b6a8-f49f-499e-853a-0a0bc2a237e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:22.865967 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f16f05-1c04-44b5-9da8-ffff4eb29eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:22.866759 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f16f05-1c04-44b5-9da8-ffff4eb29eed admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:22.867043 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f16f05-1c04-44b5-9da8-ffff4eb29eed admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:22.867365 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f16f05-1c04-44b5-9da8-ffff4eb29eed admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:22.868170 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f16f05-1c04-44b5-9da8-ffff4eb29eed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:22.869806 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f16f05-1c04-44b5-9da8-ffff4eb29eed admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:22.880170 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 801/1606] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:56:22.881256 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f16f05-1c04-44b5-9da8-ffff4eb29eed admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:22.891616 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f16f05-1c04-44b5-9da8-ffff4eb29eed admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:22.896624 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd935dd-fab5-4339-9094-d4fb9bf70089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:22.897271 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd935dd-fab5-4339-9094-d4fb9bf70089 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:22.897551 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd935dd-fab5-4339-9094-d4fb9bf70089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:22.897825 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd935dd-fab5-4339-9094-d4fb9bf70089 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:22.898659 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd935dd-fab5-4339-9094-d4fb9bf70089 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:22.900238 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd935dd-fab5-4339-9094-d4fb9bf70089 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:22.901258 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 806/1607] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:22.902097 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f87b6a8-f49f-499e-853a-0a0bc2a237e8 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:22.911712 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f87b6a8-f49f-499e-853a-0a0bc2a237e8 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:22.916733 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 802/1608] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:56:22.916733 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65600f83-cb4b-4049-aa68-d175e0748aa4 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:22.916733 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65600f83-cb4b-4049-aa68-d175e0748aa4 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:22.916733 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65600f83-cb4b-4049-aa68-d175e0748aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:22.917085 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a2c4f-27e4-4ab9-a9c9-4358d294c209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:22.917915 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a2c4f-27e4-4ab9-a9c9-4358d294c209 admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:22.918193 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a2c4f-27e4-4ab9-a9c9-4358d294c209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:22.918469 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a2c4f-27e4-4ab9-a9c9-4358d294c209 admin admin] PATH_INFO: `/v3/projects/538de7f15e7341f69d9695c7dc9005d3/users/dd524429299f4964b12612bc0659bcd9/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:22.933828 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a2c4f-27e4-4ab9-a9c9-4358d294c209 admin admin] RBAC: Authorizing `identity:create_grant(project_id=538de7f15e7341f69d9695c7dc9005d3, user_id=dd524429299f4964b12612bc0659bcd9, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:22.936601 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a2c4f-27e4-4ab9-a9c9-4358d294c209 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:22.942370 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-65600f83-cb4b-4049-aa68-d175e0748aa4 None None] MFA Rules not processed for user `62bd2ed5d8f34a429d1a78d4498647a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:22.943828 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-65600f83-cb4b-4049-aa68-d175e0748aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:22.951145 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 807/1609] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:56:22 2022] PUT /identity/v3/projects/538de7f15e7341f69d9695c7dc9005d3/users/dd524429299f4964b12612bc0659bcd9/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:56:22.952118 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a2c4f-27e4-4ab9-a9c9-4358d294c209 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:23.018429 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 803/1610] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:23.018779 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a2c4f-27e4-4ab9-a9c9-4358d294c209 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:23.019519 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65600f83-cb4b-4049-aa68-d175e0748aa4 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:23.028221 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe671a-d7bc-4499-aa9c-5b7f02017074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:23.028966 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe671a-d7bc-4499-aa9c-5b7f02017074 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:23.029421 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe671a-d7bc-4499-aa9c-5b7f02017074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:23.029748 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe671a-d7bc-4499-aa9c-5b7f02017074 admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:23.030554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fe671a-d7bc-4499-aa9c-5b7f02017074 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:23.032360 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65600f83-cb4b-4049-aa68-d175e0748aa4 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:23.032607 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fe671a-d7bc-4499-aa9c-5b7f02017074 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:23.037872 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0032b5d8-76ac-4422-83bb-1e4a6c01f87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:23.038910 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0032b5d8-76ac-4422-83bb-1e4a6c01f87e admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:23.039193 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0032b5d8-76ac-4422-83bb-1e4a6c01f87e admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:23.039523 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0032b5d8-76ac-4422-83bb-1e4a6c01f87e admin admin] PATH_INFO: `/v3/projects/c435aab218b8498585477269fe5ecc60/users/56362ad5e50c4f4e8fb694f845d02924/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:23.057091 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0032b5d8-76ac-4422-83bb-1e4a6c01f87e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c435aab218b8498585477269fe5ecc60, user_id=56362ad5e50c4f4e8fb694f845d02924, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:23.060417 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0032b5d8-76ac-4422-83bb-1e4a6c01f87e admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:23.061946 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 808/1611] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:56:22 2022] POST /identity/v3/projects => generated 405 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:56:23.063201 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff4f53-6068-42c6-bc65-9e441317dafb None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:23.063514 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff4f53-6068-42c6-bc65-9e441317dafb None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:23.063974 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff4f53-6068-42c6-bc65-9e441317dafb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:23.076810 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 804/1612] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:56:23 2022] PUT /identity/v3/projects/c435aab218b8498585477269fe5ecc60/users/56362ad5e50c4f4e8fb694f845d02924/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:56:23.081740 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0032b5d8-76ac-4422-83bb-1e4a6c01f87e admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:23.082786 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-84ff4f53-6068-42c6-bc65-9e441317dafb None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:23.084082 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-84ff4f53-6068-42c6-bc65-9e441317dafb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:23.112376 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:23.142378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 809/1613] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:23.143237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ff4f53-6068-42c6-bc65-9e441317dafb None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:23.146980 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0032b5d8-76ac-4422-83bb-1e4a6c01f87e admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:23.152063 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b661ce-9467-4bf9-a82e-9ab85c37f363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:23.152263 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:23.152899 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b661ce-9467-4bf9-a82e-9ab85c37f363 admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:23.153184 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b661ce-9467-4bf9-a82e-9ab85c37f363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:23.153462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b661ce-9467-4bf9-a82e-9ab85c37f363 admin admin] PATH_INFO: `/v3/projects/ed1b99fee624403eac4d4d016a86ba68/users/5537726541594995b03903ca768d8acc/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:23.168362 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b661ce-9467-4bf9-a82e-9ab85c37f363 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed1b99fee624403eac4d4d016a86ba68, user_id=5537726541594995b03903ca768d8acc, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:23.171028 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b661ce-9467-4bf9-a82e-9ab85c37f363 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:23.185346 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 805/1614] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:56:23 2022] PUT /identity/v3/projects/ed1b99fee624403eac4d4d016a86ba68/users/5537726541594995b03903ca768d8acc/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:56:23.186487 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b661ce-9467-4bf9-a82e-9ab85c37f363 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:23.204150 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ff4f53-6068-42c6-bc65-9e441317dafb None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:23.250746 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b661ce-9467-4bf9-a82e-9ab85c37f363 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:23.252108 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c813d-20d3-486f-a595-87700d591362 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:23.252766 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c813d-20d3-486f-a595-87700d591362 service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:23.253048 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c813d-20d3-486f-a595-87700d591362 service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:23.253321 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c813d-20d3-486f-a595-87700d591362 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:23.255570 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5187b33f-26d7-4b2e-b2bf-0d875961b33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:23.256469 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5187b33f-26d7-4b2e-b2bf-0d875961b33c admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:23.256775 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5187b33f-26d7-4b2e-b2bf-0d875961b33c admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:23.257052 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5187b33f-26d7-4b2e-b2bf-0d875961b33c admin admin] PATH_INFO: `/v3/projects/96afcc3674a54ffb86c1f0298a527e61/users/ababea3e976f46889f2b1980edfee883/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:23.263115 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856c813d-20d3-486f-a595-87700d591362 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:23.264626 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856c813d-20d3-486f-a595-87700d591362 service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:23.275236 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5187b33f-26d7-4b2e-b2bf-0d875961b33c admin admin] RBAC: Authorizing `identity:create_grant(project_id=96afcc3674a54ffb86c1f0298a527e61, user_id=ababea3e976f46889f2b1980edfee883, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:23.275848 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 810/1615] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:56:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:56:23.277365 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba09223-75ae-456a-947a-8f07d23184c4 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:23.277651 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba09223-75ae-456a-947a-8f07d23184c4 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:23.277947 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba09223-75ae-456a-947a-8f07d23184c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:23.278077 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5187b33f-26d7-4b2e-b2bf-0d875961b33c admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:23.297823 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 806/1616] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:56:23 2022] PUT /identity/v3/projects/96afcc3674a54ffb86c1f0298a527e61/users/ababea3e976f46889f2b1980edfee883/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:56:23.299564 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72394c7-5930-41a2-bef6-21f4f5ec9dd5 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:23.300205 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72394c7-5930-41a2-bef6-21f4f5ec9dd5 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:23.300626 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72394c7-5930-41a2-bef6-21f4f5ec9dd5 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:23.301852 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 807/1617] 10.222.0.26 () {54 vars in 961 bytes} [Thu Jun 23 01:56:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:56:23.304076 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72394c7-5930-41a2-bef6-21f4f5ec9dd5 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:23.313795 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-2ba09223-75ae-456a-947a-8f07d23184c4 None None] MFA Rules not processed for user `dd524429299f4964b12612bc0659bcd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:23.315144 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-2ba09223-75ae-456a-947a-8f07d23184c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:23.377978 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72394c7-5930-41a2-bef6-21f4f5ec9dd5 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:23.383319 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda97c1b-cc59-411e-83e0-c4c52c5fb1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:23.384001 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda97c1b-cc59-411e-83e0-c4c52c5fb1fc admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:23.384279 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda97c1b-cc59-411e-83e0-c4c52c5fb1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:23.384671 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda97c1b-cc59-411e-83e0-c4c52c5fb1fc admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:23.385390 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda97c1b-cc59-411e-83e0-c4c52c5fb1fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:23.387753 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda97c1b-cc59-411e-83e0-c4c52c5fb1fc admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:23.390322 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 811/1618] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:23 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:23.395732 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1cf731-b7aa-4e56-89bf-d1af418dbe4c None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:23.396034 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1cf731-b7aa-4e56-89bf-d1af418dbe4c None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:23.396318 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1cf731-b7aa-4e56-89bf-d1af418dbe4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:23.413932 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 808/1619] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:56:23 2022] POST /identity/v3/users => generated 309 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:56:23.414863 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda97c1b-cc59-411e-83e0-c4c52c5fb1fc admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:23.416128 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-4b1cf731-b7aa-4e56-89bf-d1af418dbe4c None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:23.417432 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-4b1cf731-b7aa-4e56-89bf-d1af418dbe4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:23.425947 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 812/1620] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:23.426858 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1cf731-b7aa-4e56-89bf-d1af418dbe4c None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:23.441205 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1cf731-b7aa-4e56-89bf-d1af418dbe4c None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:23.446996 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eec5bdf-9554-440b-86d3-b873a3654a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:23.447714 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec5bdf-9554-440b-86d3-b873a3654a2b admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:23.447999 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec5bdf-9554-440b-86d3-b873a3654a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:23.448539 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec5bdf-9554-440b-86d3-b873a3654a2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:23.449144 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:23.449400 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eec5bdf-9554-440b-86d3-b873a3654a2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:23.451541 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eec5bdf-9554-440b-86d3-b873a3654a2b admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:23.466700 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 813/1621] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:56:23.466700 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1832636-9bca-4a3d-b41a-f94e7be97250 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:23.466700 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1832636-9bca-4a3d-b41a-f94e7be97250 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:23.466700 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1832636-9bca-4a3d-b41a-f94e7be97250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:23.485102 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-b1832636-9bca-4a3d-b41a-f94e7be97250 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:23.486509 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda97c1b-cc59-411e-83e0-c4c52c5fb1fc admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:23.486637 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-b1832636-9bca-4a3d-b41a-f94e7be97250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:23.490244 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd057d0b-4c2d-4fb6-9497-fbd076c307dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:23.490901 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd057d0b-4c2d-4fb6-9497-fbd076c307dd service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:23.491243 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd057d0b-4c2d-4fb6-9497-fbd076c307dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:23.491987 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd057d0b-4c2d-4fb6-9497-fbd076c307dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:23.496691 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 814/1622] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:23.498154 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470693e1-9d6c-4969-94e6-dd9a91cc1885 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:23.498586 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470693e1-9d6c-4969-94e6-dd9a91cc1885 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:23.498958 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470693e1-9d6c-4969-94e6-dd9a91cc1885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:23.509049 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd057d0b-4c2d-4fb6-9497-fbd076c307dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:23.510895 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd057d0b-4c2d-4fb6-9497-fbd076c307dd service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:23.519612 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-470693e1-9d6c-4969-94e6-dd9a91cc1885 None None] MFA Rules not processed for user `ae4e83df33b64a3ebf28f54e2d94fe63`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:23.520810 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-470693e1-9d6c-4969-94e6-dd9a91cc1885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:23.524490 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 809/1623] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:56:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:56:23.525546 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd057d0b-4c2d-4fb6-9497-fbd076c307dd service neutron] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:23.549544 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:23.596336 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd057d0b-4c2d-4fb6-9497-fbd076c307dd service neutron] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:23.600009 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6a6f2-9cda-4a10-85e6-754de1dda2cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:23.600793 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6a6f2-9cda-4a10-85e6-754de1dda2cb service cinder] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:23.601137 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6a6f2-9cda-4a10-85e6-754de1dda2cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:23.601442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 815/1624] 10.222.0.26 () {58 vars in 1058 bytes} [Thu Jun 23 01:56:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:23.601563 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6a6f2-9cda-4a10-85e6-754de1dda2cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:23.602583 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470693e1-9d6c-4969-94e6-dd9a91cc1885 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:23.615705 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa6a6f2-9cda-4a10-85e6-754de1dda2cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:23.617184 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa6a6f2-9cda-4a10-85e6-754de1dda2cb service cinder] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:23.620535 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470693e1-9d6c-4969-94e6-dd9a91cc1885 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:23.627817 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 810/1625] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:56:23 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:56:23.628665 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6a6f2-9cda-4a10-85e6-754de1dda2cb service cinder] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:23.628812 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00316505-ef34-42ec-8893-fadc6a96dc21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:23.629644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00316505-ef34-42ec-8893-fadc6a96dc21 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:23.629993 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-caa6a6f2-9cda-4a10-85e6-754de1dda2cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:23.630117 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00316505-ef34-42ec-8893-fadc6a96dc21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:23.630460 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00316505-ef34-42ec-8893-fadc6a96dc21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:23.631423 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00316505-ef34-42ec-8893-fadc6a96dc21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:23.633779 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00316505-ef34-42ec-8893-fadc6a96dc21 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:23.641873 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6a6f2-9cda-4a10-85e6-754de1dda2cb service cinder] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:23.646582 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f526e3-852d-4fa7-895e-408e85ad13c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:23.650283 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f526e3-852d-4fa7-895e-408e85ad13c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:23.650435 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 816/1626] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:56:23.650561 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f526e3-852d-4fa7-895e-408e85ad13c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:23.650561 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f526e3-852d-4fa7-895e-408e85ad13c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:23.650729 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00316505-ef34-42ec-8893-fadc6a96dc21 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:23.660068 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00316505-ef34-42ec-8893-fadc6a96dc21 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:23.660829 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f526e3-852d-4fa7-895e-408e85ad13c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:23.662414 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f526e3-852d-4fa7-895e-408e85ad13c1 service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:23.665313 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00969a3d-819c-44bc-ade2-552b80ea344d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:23.666701 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00969a3d-819c-44bc-ade2-552b80ea344d admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:23.667118 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00969a3d-819c-44bc-ade2-552b80ea344d admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:23.667488 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00969a3d-819c-44bc-ade2-552b80ea344d admin admin] PATH_INFO: `/v3/projects/ed1b99fee624403eac4d4d016a86ba68/users/5537726541594995b03903ca768d8acc/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:23.669857 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00969a3d-819c-44bc-ade2-552b80ea344d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed1b99fee624403eac4d4d016a86ba68, user_id=5537726541594995b03903ca768d8acc, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:23.672767 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00969a3d-819c-44bc-ade2-552b80ea344d admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:23.684718 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 811/1627] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:56:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:56:23.686971 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f526e3-852d-4fa7-895e-408e85ad13c1 service neutron] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:23.691949 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 817/1628] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:56:23 2022] PUT /identity/v3/projects/ed1b99fee624403eac4d4d016a86ba68/users/5537726541594995b03903ca768d8acc/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:56:23.693127 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00969a3d-819c-44bc-ade2-552b80ea344d admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:23.706885 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f526e3-852d-4fa7-895e-408e85ad13c1 service neutron] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:23.819694 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3780c40-fd6c-4995-8d65-3325f17afd8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:23.820983 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3780c40-fd6c-4995-8d65-3325f17afd8a service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:23.821548 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3780c40-fd6c-4995-8d65-3325f17afd8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:23.821917 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3780c40-fd6c-4995-8d65-3325f17afd8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:23.829141 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00969a3d-819c-44bc-ade2-552b80ea344d admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:23.848349 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0da4707-14df-4beb-b85c-ebd938cbd180 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4e83df33b64a3ebf28f54e2d94fe63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:23.849027 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da4707-14df-4beb-b85c-ebd938cbd180 service nova] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:23.849319 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da4707-14df-4beb-b85c-ebd938cbd180 service nova] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:23.849597 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da4707-14df-4beb-b85c-ebd938cbd180 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:23.933914 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3780c40-fd6c-4995-8d65-3325f17afd8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:23.935700 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3780c40-fd6c-4995-8d65-3325f17afd8a service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:23.943409 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0da4707-14df-4beb-b85c-ebd938cbd180 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:23.946416 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0da4707-14df-4beb-b85c-ebd938cbd180 service nova] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:23.956820 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 812/1629] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:56:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:56:23.958041 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3780c40-fd6c-4995-8d65-3325f17afd8a service neutron] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:23.981472 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:23.986235 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3780c40-fd6c-4995-8d65-3325f17afd8a service neutron] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:23.995386 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 818/1630] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:56:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:56:24.001227 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add10476-6e74-4da8-90d4-e650354529c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:24.002055 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add10476-6e74-4da8-90d4-e650354529c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:24.002331 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add10476-6e74-4da8-90d4-e650354529c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:24.002635 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add10476-6e74-4da8-90d4-e650354529c3 admin admin] PATH_INFO: `/v3/projects/fd74075e7353488a8795be0382704fa4/users/47ed6f72b360446b8f1e834f3a81a59b/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:24.033322 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add10476-6e74-4da8-90d4-e650354529c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd74075e7353488a8795be0382704fa4, user_id=47ed6f72b360446b8f1e834f3a81a59b, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:24.036126 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add10476-6e74-4da8-90d4-e650354529c3 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:24.067163 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 813/1631] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:56:23 2022] PUT /identity/v3/projects/fd74075e7353488a8795be0382704fa4/users/47ed6f72b360446b8f1e834f3a81a59b/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:56:24.068456 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0da4707-14df-4beb-b85c-ebd938cbd180 service nova] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:24.082517 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0da4707-14df-4beb-b85c-ebd938cbd180 service nova] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:24.186904 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:24.255367 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa738caa-77b1-4e83-9cf9-80a74007bf20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:24.256147 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa738caa-77b1-4e83-9cf9-80a74007bf20 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:24.256420 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa738caa-77b1-4e83-9cf9-80a74007bf20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:24.256713 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa738caa-77b1-4e83-9cf9-80a74007bf20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:24.258259 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa738caa-77b1-4e83-9cf9-80a74007bf20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:24.260087 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa738caa-77b1-4e83-9cf9-80a74007bf20 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:24.291511 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 819/1632] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:56:24.298786 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add10476-6e74-4da8-90d4-e650354529c3 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:24.311006 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add10476-6e74-4da8-90d4-e650354529c3 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:24.325641 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ec57b-61af-41e8-866b-69127e994dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:24.327104 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ec57b-61af-41e8-866b-69127e994dd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:24.327716 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ec57b-61af-41e8-866b-69127e994dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:24.328362 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ec57b-61af-41e8-866b-69127e994dd4 admin admin] PATH_INFO: `/v3/projects/fd74075e7353488a8795be0382704fa4/users/47ed6f72b360446b8f1e834f3a81a59b/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:24.331760 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7ec57b-61af-41e8-866b-69127e994dd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd74075e7353488a8795be0382704fa4, user_id=47ed6f72b360446b8f1e834f3a81a59b, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:24.334746 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7ec57b-61af-41e8-866b-69127e994dd4 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:24.363192 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 814/1633] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:56:24 2022] PUT /identity/v3/projects/fd74075e7353488a8795be0382704fa4/users/47ed6f72b360446b8f1e834f3a81a59b/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:56:24.374283 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33aa2e3-2dbe-4ff8-949a-c186dd55c64f None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:24.374575 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33aa2e3-2dbe-4ff8-949a-c186dd55c64f None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:24.374857 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33aa2e3-2dbe-4ff8-949a-c186dd55c64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:24.386834 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e192d7-a2ff-4f7d-a80b-00f33ede5676 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:24.387434 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e192d7-a2ff-4f7d-a80b-00f33ede5676 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:24.388517 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e192d7-a2ff-4f7d-a80b-00f33ede5676 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:24.389963 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 815/1634] 10.222.0.26 () {54 vars in 912 bytes} [Thu Jun 23 01:56:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:56:24.396686 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac7cff-fe35-40dd-908f-791550b1daca None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:24.397393 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac7cff-fe35-40dd-908f-791550b1daca None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:24.397854 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac7cff-fe35-40dd-908f-791550b1daca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:24.412123 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-c33aa2e3-2dbe-4ff8-949a-c186dd55c64f None None] MFA Rules not processed for user `47ed6f72b360446b8f1e834f3a81a59b`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:24.414328 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-c33aa2e3-2dbe-4ff8-949a-c186dd55c64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:24.461722 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-53ac7cff-fe35-40dd-908f-791550b1daca None None] MFA Rules not processed for user `ae4e83df33b64a3ebf28f54e2d94fe63`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:24.463851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-53ac7cff-fe35-40dd-908f-791550b1daca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:24.576383 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 820/1635] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:24 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:24.583175 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 816/1636] 10.222.0.26 () {58 vars in 1009 bytes} [Thu Jun 23 01:56:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:24.590595 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33aa2e3-2dbe-4ff8-949a-c186dd55c64f None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:24.660962 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33aa2e3-2dbe-4ff8-949a-c186dd55c64f None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:24.665152 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7554b0-1f5d-4733-b5ff-1d92c699ab7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:24.665951 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7554b0-1f5d-4733-b5ff-1d92c699ab7e service cinder] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:24.666490 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7554b0-1f5d-4733-b5ff-1d92c699ab7e service cinder] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:24.667093 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7554b0-1f5d-4733-b5ff-1d92c699ab7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:24.677011 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac7cff-fe35-40dd-908f-791550b1daca None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:24.680272 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7554b0-1f5d-4733-b5ff-1d92c699ab7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:24.682286 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7554b0-1f5d-4733-b5ff-1d92c699ab7e service cinder] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:24.691627 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac7cff-fe35-40dd-908f-791550b1daca None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:24.696715 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 821/1637] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:56:24 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:56:24.713212 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43321db7-d16d-46b7-82fd-164e80d34def service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:24.715523 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43321db7-d16d-46b7-82fd-164e80d34def service cinder] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:24.715914 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43321db7-d16d-46b7-82fd-164e80d34def service cinder] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:24.716733 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43321db7-d16d-46b7-82fd-164e80d34def service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:24.728954 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43321db7-d16d-46b7-82fd-164e80d34def service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:24.730556 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43321db7-d16d-46b7-82fd-164e80d34def service cinder] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:24.743269 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 817/1638] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:56:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:56:25.035484 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7554b0-1f5d-4733-b5ff-1d92c699ab7e service cinder] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:25.166333 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7554b0-1f5d-4733-b5ff-1d92c699ab7e service cinder] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:25.181439 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1941c90e-1daf-4ac4-b61f-1d118be10c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:25.186333 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1941c90e-1daf-4ac4-b61f-1d118be10c85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:25.186683 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1941c90e-1daf-4ac4-b61f-1d118be10c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:25.187096 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1941c90e-1daf-4ac4-b61f-1d118be10c85 admin admin] PATH_INFO: `/v3/users/11551dbcae304eadb0c2252a300346b3` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:25.188749 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1941c90e-1daf-4ac4-b61f-1d118be10c85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11551dbcae304eadb0c2252a300346b3)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:25.191119 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1941c90e-1daf-4ac4-b61f-1d118be10c85 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:25.193924 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:25.286581 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-1941c90e-1daf-4ac4-b61f-1d118be10c85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11551dbcae304eadb0c2252a300346b3'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:25.321607 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-1941c90e-1daf-4ac4-b61f-1d118be10c85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11551dbcae304eadb0c2252a300346b3'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:25.346581 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-1941c90e-1daf-4ac4-b61f-1d118be10c85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11551dbcae304eadb0c2252a300346b3'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:25.370285 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-1941c90e-1daf-4ac4-b61f-1d118be10c85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11551dbcae304eadb0c2252a300346b3'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:25.371261 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-1941c90e-1daf-4ac4-b61f-1d118be10c85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11551dbcae304eadb0c2252a300346b3'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:25.382146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 822/1639] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:56:25 2022] DELETE /identity/v3/users/11551dbcae304eadb0c2252a300346b3 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:56:25.386861 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de9b6d-7dc5-4dd3-b1df-d15a02dd4f41 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:25.387349 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de9b6d-7dc5-4dd3-b1df-d15a02dd4f41 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:25.387725 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de9b6d-7dc5-4dd3-b1df-d15a02dd4f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:25.419443 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-64de9b6d-7dc5-4dd3-b1df-d15a02dd4f41 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:25.419443 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-64de9b6d-7dc5-4dd3-b1df-d15a02dd4f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:25.512249 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 818/1640] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:25.560074 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1941c90e-1daf-4ac4-b61f-1d118be10c85 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:25.562449 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-1941c90e-1daf-4ac4-b61f-1d118be10c85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:25.716222 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1941c90e-1daf-4ac4-b61f-1d118be10c85 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:25.726984 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0eb716-b3f6-4ff1-ba00-e4cf36b05089 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:25.729133 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0eb716-b3f6-4ff1-ba00-e4cf36b05089 service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:25.729506 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0eb716-b3f6-4ff1-ba00-e4cf36b05089 service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:25.729822 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0eb716-b3f6-4ff1-ba00-e4cf36b05089 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:25.743938 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0eb716-b3f6-4ff1-ba00-e4cf36b05089 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:25.757586 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0eb716-b3f6-4ff1-ba00-e4cf36b05089 service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:25.795999 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 823/1641] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:56:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:56:25.986107 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b14125-55e8-46e1-a036-54b9bf4d8535 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:25.986643 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b14125-55e8-46e1-a036-54b9bf4d8535 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:25.987034 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b14125-55e8-46e1-a036-54b9bf4d8535 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:25.988546 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 819/1642] 10.222.0.26 () {54 vars in 961 bytes} [Thu Jun 23 01:56:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:56:25.998733 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16936eaa-cd72-46da-a669-797e0f6e2558 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:25.999061 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16936eaa-cd72-46da-a669-797e0f6e2558 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:25.999335 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16936eaa-cd72-46da-a669-797e0f6e2558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:26.027644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-16936eaa-cd72-46da-a669-797e0f6e2558 None None] MFA Rules not processed for user `ae4e83df33b64a3ebf28f54e2d94fe63`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:26.029029 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-16936eaa-cd72-46da-a669-797e0f6e2558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:26.142961 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 824/1643] 10.222.0.26 () {58 vars in 1058 bytes} [Thu Jun 23 01:56:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:26.144267 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5058ff8-0e51-4a85-8f92-7da02f9383fd None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:26.145012 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5058ff8-0e51-4a85-8f92-7da02f9383fd None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:26.145012 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5058ff8-0e51-4a85-8f92-7da02f9383fd None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:26.146574 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 820/1644] 10.222.0.26 () {54 vars in 912 bytes} [Thu Jun 23 01:56:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:56:26.166489 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06127eef-ca1a-4d2c-8162-e869c5c8d5a2 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:26.167766 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06127eef-ca1a-4d2c-8162-e869c5c8d5a2 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:26.169580 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06127eef-ca1a-4d2c-8162-e869c5c8d5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:26.194656 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-06127eef-ca1a-4d2c-8162-e869c5c8d5a2 None None] MFA Rules not processed for user `ae4e83df33b64a3ebf28f54e2d94fe63`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:26.196217 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-06127eef-ca1a-4d2c-8162-e869c5c8d5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:26.207991 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 825/1645] 10.222.0.26 () {58 vars in 1009 bytes} [Thu Jun 23 01:56:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:26.222427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5058ff8-0e51-4a85-8f92-7da02f9383fd None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:26.224393 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-b5058ff8-0e51-4a85-8f92-7da02f9383fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:26.352750 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:26.371271 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5058ff8-0e51-4a85-8f92-7da02f9383fd None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:26.379598 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d9225-9daf-4b3b-8e20-8bd7bedd40bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:26.381055 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d9225-9daf-4b3b-8e20-8bd7bedd40bc service cinder] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:26.381688 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d9225-9daf-4b3b-8e20-8bd7bedd40bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:26.383037 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6d9225-9daf-4b3b-8e20-8bd7bedd40bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:26.397382 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6d9225-9daf-4b3b-8e20-8bd7bedd40bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:26.401308 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6d9225-9daf-4b3b-8e20-8bd7bedd40bc service cinder] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:26.423681 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 821/1646] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:56:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:56:26.942566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b4086-49d3-4ffe-82a9-da52e76a7f3c None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:26.943199 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b4086-49d3-4ffe-82a9-da52e76a7f3c None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:26.943701 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b4086-49d3-4ffe-82a9-da52e76a7f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:26.957946 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d9225-9daf-4b3b-8e20-8bd7bedd40bc service cinder] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:26.959654 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-0b6d9225-9daf-4b3b-8e20-8bd7bedd40bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:26.978565 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:26.994398 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6d9225-9daf-4b3b-8e20-8bd7bedd40bc service cinder] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:27.019493 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86230e87-ced2-4b2f-a150-cfedce383fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:27.020507 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86230e87-ced2-4b2f-a150-cfedce383fb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:27.020918 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86230e87-ced2-4b2f-a150-cfedce383fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:27.021449 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86230e87-ced2-4b2f-a150-cfedce383fb6 admin admin] PATH_INFO: `/v3/projects/e4537bc7993743cabfa3752ab1489fed` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:27.023772 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86230e87-ced2-4b2f-a150-cfedce383fb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4537bc7993743cabfa3752ab1489fed)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:27.028527 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86230e87-ced2-4b2f-a150-cfedce383fb6 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:27.032262 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-749b4086-49d3-4ffe-82a9-da52e76a7f3c None None] MFA Rules not processed for user `5537726541594995b03903ca768d8acc`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:27.032655 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-749b4086-49d3-4ffe-82a9-da52e76a7f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:27.196563 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-86230e87-ced2-4b2f-a150-cfedce383fb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4537bc7993743cabfa3752ab1489fed', 'request_id': 'req-86230e87-ced2-4b2f-a150-cfedce383fb6'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:27.218749 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-86230e87-ced2-4b2f-a150-cfedce383fb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4537bc7993743cabfa3752ab1489fed', 'request_id': 'req-86230e87-ced2-4b2f-a150-cfedce383fb6'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:27.234549 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-86230e87-ced2-4b2f-a150-cfedce383fb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4537bc7993743cabfa3752ab1489fed', 'request_id': 'req-86230e87-ced2-4b2f-a150-cfedce383fb6'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:27.257144 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-86230e87-ced2-4b2f-a150-cfedce383fb6 admin admin] The token cache is being invalidate because project e4537bc7993743cabfa3752ab1489fed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:56:27.257734 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-86230e87-ced2-4b2f-a150-cfedce383fb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:27.258914 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-86230e87-ced2-4b2f-a150-cfedce383fb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:27.263524 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 822/1647] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:56:26 2022] DELETE /identity/v3/projects/e4537bc7993743cabfa3752ab1489fed => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 01:56:27.268435 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86230e87-ced2-4b2f-a150-cfedce383fb6 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:27.273486 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 826/1648] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:26 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:27.273645 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-86230e87-ced2-4b2f-a150-cfedce383fb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:27.395439 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86230e87-ced2-4b2f-a150-cfedce383fb6 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:27.420495 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d320ce9-f97c-4f32-9b62-5bcacac5b5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:27.422197 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d320ce9-f97c-4f32-9b62-5bcacac5b5d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:27.422664 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d320ce9-f97c-4f32-9b62-5bcacac5b5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:27.423117 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d320ce9-f97c-4f32-9b62-5bcacac5b5d4 admin admin] PATH_INFO: `/v3/users/47ed6f72b360446b8f1e834f3a81a59b` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:27.425838 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d320ce9-f97c-4f32-9b62-5bcacac5b5d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47ed6f72b360446b8f1e834f3a81a59b)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:27.427390 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: WARNING py.warnings [None req-1d320ce9-f97c-4f32-9b62-5bcacac5b5d4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 01:56:27.427390 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: warnings.warn(msg) Jun 23 01:56:27.427390 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: Jun 23 01:56:27.429287 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d320ce9-f97c-4f32-9b62-5bcacac5b5d4 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:27.558531 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-1d320ce9-f97c-4f32-9b62-5bcacac5b5d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47ed6f72b360446b8f1e834f3a81a59b'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:27.605526 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-1d320ce9-f97c-4f32-9b62-5bcacac5b5d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47ed6f72b360446b8f1e834f3a81a59b'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:27.624374 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-1d320ce9-f97c-4f32-9b62-5bcacac5b5d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47ed6f72b360446b8f1e834f3a81a59b'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:27.631184 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:27.651676 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-1d320ce9-f97c-4f32-9b62-5bcacac5b5d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47ed6f72b360446b8f1e834f3a81a59b'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:27.652968 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-1d320ce9-f97c-4f32-9b62-5bcacac5b5d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47ed6f72b360446b8f1e834f3a81a59b'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:27.664544 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 823/1649] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:56:27 2022] DELETE /identity/v3/users/47ed6f72b360446b8f1e834f3a81a59b => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:56:29.470412 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b5811d-beb1-42f1-8e41-30ff614448bd None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:29.470922 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b5811d-beb1-42f1-8e41-30ff614448bd None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:29.471632 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b5811d-beb1-42f1-8e41-30ff614448bd None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:29.473427 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 827/1650] 10.222.0.26 () {54 vars in 921 bytes} [Thu Jun 23 01:56:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:56:29.494904 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c45b0-2c91-4168-bd00-8f7f5478d6b5 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:29.495481 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c45b0-2c91-4168-bd00-8f7f5478d6b5 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:29.495481 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c45b0-2c91-4168-bd00-8f7f5478d6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:29.544282 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-d76c45b0-2c91-4168-bd00-8f7f5478d6b5 None None] MFA Rules not processed for user `e5bf3baee1b843fa89b65e5485f6bebb`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:29.546616 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-d76c45b0-2c91-4168-bd00-8f7f5478d6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:29.795921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 824/1651] 10.222.0.26 () {58 vars in 1018 bytes} [Thu Jun 23 01:56:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 01:56:29.845861 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b5811d-beb1-42f1-8e41-30ff614448bd None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:29.848376 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-98b5811d-beb1-42f1-8e41-30ff614448bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:29.895711 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b5811d-beb1-42f1-8e41-30ff614448bd None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:29.917922 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606aac36-bf06-4b37-a670-7ea2493e0eb1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5bf3baee1b843fa89b65e5485f6bebb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:29.918876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606aac36-bf06-4b37-a670-7ea2493e0eb1 service placement] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:29.919365 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606aac36-bf06-4b37-a670-7ea2493e0eb1 service placement] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:29.920235 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606aac36-bf06-4b37-a670-7ea2493e0eb1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:29.946845 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606aac36-bf06-4b37-a670-7ea2493e0eb1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:29.948746 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606aac36-bf06-4b37-a670-7ea2493e0eb1 service placement] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:29.989320 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 828/1652] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:56:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:56:30.160537 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c45b0-2c91-4168-bd00-8f7f5478d6b5 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:30.176894 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-d76c45b0-2c91-4168-bd00-8f7f5478d6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:30.420738 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c45b0-2c91-4168-bd00-8f7f5478d6b5 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:30.436791 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6e758c-2046-4f88-ac0e-99d36bc897ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:30.458611 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e758c-2046-4f88-ac0e-99d36bc897ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:30.459025 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e758c-2046-4f88-ac0e-99d36bc897ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:30.459408 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e758c-2046-4f88-ac0e-99d36bc897ab admin admin] PATH_INFO: `/v3/projects/fd74075e7353488a8795be0382704fa4` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:30.464469 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6e758c-2046-4f88-ac0e-99d36bc897ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd74075e7353488a8795be0382704fa4)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:30.472368 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6e758c-2046-4f88-ac0e-99d36bc897ab admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:30.669583 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-1e6e758c-2046-4f88-ac0e-99d36bc897ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd74075e7353488a8795be0382704fa4', 'request_id': 'req-1e6e758c-2046-4f88-ac0e-99d36bc897ab'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:30.682064 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-1e6e758c-2046-4f88-ac0e-99d36bc897ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd74075e7353488a8795be0382704fa4', 'request_id': 'req-1e6e758c-2046-4f88-ac0e-99d36bc897ab'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:30.699918 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-1e6e758c-2046-4f88-ac0e-99d36bc897ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd74075e7353488a8795be0382704fa4', 'request_id': 'req-1e6e758c-2046-4f88-ac0e-99d36bc897ab'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:30.739986 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-1e6e758c-2046-4f88-ac0e-99d36bc897ab admin admin] The token cache is being invalidate because project fd74075e7353488a8795be0382704fa4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:56:30.740633 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-1e6e758c-2046-4f88-ac0e-99d36bc897ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:30.741491 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-1e6e758c-2046-4f88-ac0e-99d36bc897ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:30.756940 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 825/1653] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:56:30 2022] DELETE /identity/v3/projects/fd74075e7353488a8795be0382704fa4 => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:56:30.757846 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c085a90-c8ee-4c87-9ca7-c8a07e2e5858 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:30.758285 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c085a90-c8ee-4c87-9ca7-c8a07e2e5858 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:30.758758 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c085a90-c8ee-4c87-9ca7-c8a07e2e5858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:30.808721 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-2c085a90-c8ee-4c87-9ca7-c8a07e2e5858 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:30.811019 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-2c085a90-c8ee-4c87-9ca7-c8a07e2e5858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:31.001077 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 829/1654] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:31.009443 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0bec9-16ab-48a5-a2f5-2795b4085eeb None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:31.016101 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0bec9-16ab-48a5-a2f5-2795b4085eeb None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:31.016538 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0bec9-16ab-48a5-a2f5-2795b4085eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:31.068312 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-10f0bec9-16ab-48a5-a2f5-2795b4085eeb None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:31.075725 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-10f0bec9-16ab-48a5-a2f5-2795b4085eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:31.099194 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 826/1655] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:31.100276 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c085a90-c8ee-4c87-9ca7-c8a07e2e5858 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:31.116646 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c085a90-c8ee-4c87-9ca7-c8a07e2e5858 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:31.134168 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc3952-be76-460f-bce6-44fef93d732d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:31.134931 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc3952-be76-460f-bce6-44fef93d732d admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:31.135316 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc3952-be76-460f-bce6-44fef93d732d admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:31.135778 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc3952-be76-460f-bce6-44fef93d732d admin admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:31.137003 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfc3952-be76-460f-bce6-44fef93d732d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:56:31.137393 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfc3952-be76-460f-bce6-44fef93d732d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:31.139852 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfc3952-be76-460f-bce6-44fef93d732d admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:31.169328 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 830/1656] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 01:56:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:56:31.173802 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f0bec9-16ab-48a5-a2f5-2795b4085eeb None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:31.193362 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f0bec9-16ab-48a5-a2f5-2795b4085eeb None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:31.205883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b887a0-492e-4065-957d-1e8abfa6b650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:31.206803 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b887a0-492e-4065-957d-1e8abfa6b650 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:31.207249 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b887a0-492e-4065-957d-1e8abfa6b650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:31.207653 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b887a0-492e-4065-957d-1e8abfa6b650 admin admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:31.210453 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b887a0-492e-4065-957d-1e8abfa6b650 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:31.229340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b887a0-492e-4065-957d-1e8abfa6b650 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:31.246627 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:31.288970 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 827/1657] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:56:31 2022] POST /identity/v3/projects => generated 371 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:56:31.295198 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc3952-be76-460f-bce6-44fef93d732d admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:31.317557 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc3952-be76-460f-bce6-44fef93d732d admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:31.472620 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6811d0f0-f7a0-4f16-9d5b-4b4e151bac4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:31.473479 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811d0f0-f7a0-4f16-9d5b-4b4e151bac4e admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:31.474494 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811d0f0-f7a0-4f16-9d5b-4b4e151bac4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:31.474771 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811d0f0-f7a0-4f16-9d5b-4b4e151bac4e admin admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:31.475545 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6811d0f0-f7a0-4f16-9d5b-4b4e151bac4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:31.477494 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6811d0f0-f7a0-4f16-9d5b-4b4e151bac4e admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:31.521443 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:31.552700 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 831/1658] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:56:31 2022] POST /identity/v3/users => generated 286 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:56:31.555944 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b887a0-492e-4065-957d-1e8abfa6b650 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:31.569423 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b887a0-492e-4065-957d-1e8abfa6b650 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:31.585444 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e5c60-0607-4eb3-94e2-224ce6af9ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:31.586145 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e5c60-0607-4eb3-94e2-224ce6af9ec7 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:31.586423 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e5c60-0607-4eb3-94e2-224ce6af9ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:31.586688 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891e5c60-0607-4eb3-94e2-224ce6af9ec7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:31.592791 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891e5c60-0607-4eb3-94e2-224ce6af9ec7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:31.594448 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891e5c60-0607-4eb3-94e2-224ce6af9ec7 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:31.621526 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 828/1659] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:56:31.625678 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6811d0f0-f7a0-4f16-9d5b-4b4e151bac4e admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:31.639584 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6811d0f0-f7a0-4f16-9d5b-4b4e151bac4e admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:31.647775 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd139af5-a64c-4bbc-b142-1777d17014de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:31.648883 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd139af5-a64c-4bbc-b142-1777d17014de admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:31.648883 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd139af5-a64c-4bbc-b142-1777d17014de admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:31.649972 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd139af5-a64c-4bbc-b142-1777d17014de admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:31.650739 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd139af5-a64c-4bbc-b142-1777d17014de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:31.656697 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd139af5-a64c-4bbc-b142-1777d17014de admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:31.722046 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e5c60-0607-4eb3-94e2-224ce6af9ec7 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:31.731771 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 832/1660] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:56:31.737059 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891e5c60-0607-4eb3-94e2-224ce6af9ec7 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:31.750321 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa878a0-54c4-40e8-94c1-de3d8b72e2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:31.754630 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa878a0-54c4-40e8-94c1-de3d8b72e2c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:31.755254 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa878a0-54c4-40e8-94c1-de3d8b72e2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:31.756263 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa878a0-54c4-40e8-94c1-de3d8b72e2c2 admin admin] PATH_INFO: `/v3/projects/e8f79e76a31742019a5e1a8ce805106f/users/559cc0b000414c159653f9625b4fc3e1/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:31.800782 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa878a0-54c4-40e8-94c1-de3d8b72e2c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8f79e76a31742019a5e1a8ce805106f, user_id=559cc0b000414c159653f9625b4fc3e1, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:31.805635 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa878a0-54c4-40e8-94c1-de3d8b72e2c2 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:31.863012 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 829/1661] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:56:31 2022] PUT /identity/v3/projects/e8f79e76a31742019a5e1a8ce805106f/users/559cc0b000414c159653f9625b4fc3e1/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:56:31.865380 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120406e-77e2-4661-b4b3-d9629f5d1fcc None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:31.866068 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120406e-77e2-4661-b4b3-d9629f5d1fcc None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:31.866068 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120406e-77e2-4661-b4b3-d9629f5d1fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:31.950413 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-f120406e-77e2-4661-b4b3-d9629f5d1fcc None None] MFA Rules not processed for user `559cc0b000414c159653f9625b4fc3e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:31.952389 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-f120406e-77e2-4661-b4b3-d9629f5d1fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:32.261840 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 833/1662] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 398 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:32.267017 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa878a0-54c4-40e8-94c1-de3d8b72e2c2 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:32.422183 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa878a0-54c4-40e8-94c1-de3d8b72e2c2 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:32.436882 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708b590-f5ac-4e40-bb29-7bbf05780e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:32.437575 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708b590-f5ac-4e40-bb29-7bbf05780e6d admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:32.437853 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708b590-f5ac-4e40-bb29-7bbf05780e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:32.438125 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708b590-f5ac-4e40-bb29-7bbf05780e6d admin admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:32.438832 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6708b590-f5ac-4e40-bb29-7bbf05780e6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:32.440808 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6708b590-f5ac-4e40-bb29-7bbf05780e6d admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:32.499381 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 830/1663] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:56:32 2022] POST /identity/v3/projects => generated 373 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:56:32.503525 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120406e-77e2-4661-b4b3-d9629f5d1fcc None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:32.521107 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120406e-77e2-4661-b4b3-d9629f5d1fcc None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:32.707647 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc4722-de87-4f8b-81d3-4598080d8120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:32.709373 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc4722-de87-4f8b-81d3-4598080d8120 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:32.710269 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc4722-de87-4f8b-81d3-4598080d8120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:32.710548 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc4722-de87-4f8b-81d3-4598080d8120 admin admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:32.711319 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcc4722-de87-4f8b-81d3-4598080d8120 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:32.717816 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcc4722-de87-4f8b-81d3-4598080d8120 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:32.763849 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 834/1664] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:56:32 2022] POST /identity/v3/users => generated 293 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 01:56:32.767885 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708b590-f5ac-4e40-bb29-7bbf05780e6d admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:32.773236 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc4722-de87-4f8b-81d3-4598080d8120 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:32.774952 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-bbcc4722-de87-4f8b-81d3-4598080d8120 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:32.784423 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708b590-f5ac-4e40-bb29-7bbf05780e6d admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:32.793324 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc4722-de87-4f8b-81d3-4598080d8120 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:32.801916 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188d0b74-03f2-402e-bc6a-3670e698646e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:32.803172 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188d0b74-03f2-402e-bc6a-3670e698646e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:32.803614 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188d0b74-03f2-402e-bc6a-3670e698646e admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:32.804057 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188d0b74-03f2-402e-bc6a-3670e698646e admin admin] PATH_INFO: `/v3/users/b31674ccb4a24bb58ef352113d00a64a` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:32.804923 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550f731e-7f73-48ad-b662-695320fcc2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:32.805679 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188d0b74-03f2-402e-bc6a-3670e698646e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b31674ccb4a24bb58ef352113d00a64a)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:32.809725 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188d0b74-03f2-402e-bc6a-3670e698646e admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:32.811571 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f731e-7f73-48ad-b662-695320fcc2ed admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:32.812124 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f731e-7f73-48ad-b662-695320fcc2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:32.812466 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f731e-7f73-48ad-b662-695320fcc2ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:32.813553 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550f731e-7f73-48ad-b662-695320fcc2ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:32.815583 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550f731e-7f73-48ad-b662-695320fcc2ed admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:32.849788 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 831/1665] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:56:32 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:56:32.854012 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550f731e-7f73-48ad-b662-695320fcc2ed admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:32.881288 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550f731e-7f73-48ad-b662-695320fcc2ed admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:32.895095 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62565e96-7f1d-4f14-8de5-28b2a352077a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:32.896252 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62565e96-7f1d-4f14-8de5-28b2a352077a admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:32.896698 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62565e96-7f1d-4f14-8de5-28b2a352077a admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:32.897347 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62565e96-7f1d-4f14-8de5-28b2a352077a admin admin] PATH_INFO: `/v3/projects/47edb9244af243ac8ce34bb543bc319b/users/7b9053c8f3f24810a8b96601193d7f9f/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:32.934897 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-188d0b74-03f2-402e-bc6a-3670e698646e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b31674ccb4a24bb58ef352113d00a64a'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:32.944563 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62565e96-7f1d-4f14-8de5-28b2a352077a admin admin] RBAC: Authorizing `identity:create_grant(project_id=47edb9244af243ac8ce34bb543bc319b, user_id=7b9053c8f3f24810a8b96601193d7f9f, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:32.949815 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62565e96-7f1d-4f14-8de5-28b2a352077a admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:32.973084 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:32.983868 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 832/1666] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:56:32 2022] PUT /identity/v3/projects/47edb9244af243ac8ce34bb543bc319b/users/7b9053c8f3f24810a8b96601193d7f9f/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:56:32.988727 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62565e96-7f1d-4f14-8de5-28b2a352077a admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:32.994086 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-188d0b74-03f2-402e-bc6a-3670e698646e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b31674ccb4a24bb58ef352113d00a64a'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:33.006876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-188d0b74-03f2-402e-bc6a-3670e698646e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b31674ccb4a24bb58ef352113d00a64a'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:33.019713 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-188d0b74-03f2-402e-bc6a-3670e698646e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b31674ccb4a24bb58ef352113d00a64a'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:33.020797 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-188d0b74-03f2-402e-bc6a-3670e698646e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b31674ccb4a24bb58ef352113d00a64a'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:33.033997 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 835/1667] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:56:32 2022] DELETE /identity/v3/users/b31674ccb4a24bb58ef352113d00a64a => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 01:56:33.125473 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62565e96-7f1d-4f14-8de5-28b2a352077a admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:33.142597 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:33.287285 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc4e6af-f108-478d-a0be-011ebfc4411a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:33.288001 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4e6af-f108-478d-a0be-011ebfc4411a admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:33.288153 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4e6af-f108-478d-a0be-011ebfc4411a admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:33.288428 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4e6af-f108-478d-a0be-011ebfc4411a admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:33.289237 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc4e6af-f108-478d-a0be-011ebfc4411a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:33.320562 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc4e6af-f108-478d-a0be-011ebfc4411a admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:33.338493 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 833/1668] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:56:32 2022] GET /identity/v3/roles => generated 1420 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:56:33.342646 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188d0b74-03f2-402e-bc6a-3670e698646e admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:33.344325 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-188d0b74-03f2-402e-bc6a-3670e698646e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:33.364540 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188d0b74-03f2-402e-bc6a-3670e698646e admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:33.389038 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f07e03d-b0ef-4a88-ada1-76ee83b0bf2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:33.390136 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f07e03d-b0ef-4a88-ada1-76ee83b0bf2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:33.390423 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f07e03d-b0ef-4a88-ada1-76ee83b0bf2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:33.390700 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f07e03d-b0ef-4a88-ada1-76ee83b0bf2e admin admin] PATH_INFO: `/v3/projects/47edb9244af243ac8ce34bb543bc319b/users/7b9053c8f3f24810a8b96601193d7f9f/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:33.396694 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f07e03d-b0ef-4a88-ada1-76ee83b0bf2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=47edb9244af243ac8ce34bb543bc319b, user_id=7b9053c8f3f24810a8b96601193d7f9f, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:33.399741 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f07e03d-b0ef-4a88-ada1-76ee83b0bf2e admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:33.435966 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 836/1669] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:56:33 2022] PUT /identity/v3/projects/47edb9244af243ac8ce34bb543bc319b/users/7b9053c8f3f24810a8b96601193d7f9f/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 01:56:33.440406 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff84fb-c264-4abd-b632-61800aa3ca9e None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:33.440924 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff84fb-c264-4abd-b632-61800aa3ca9e None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:33.441203 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff84fb-c264-4abd-b632-61800aa3ca9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:33.457012 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f07e03d-b0ef-4a88-ada1-76ee83b0bf2e admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:33.460197 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-9f07e03d-b0ef-4a88-ada1-76ee83b0bf2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:33.508655 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-9fff84fb-c264-4abd-b632-61800aa3ca9e None None] MFA Rules not processed for user `7b9053c8f3f24810a8b96601193d7f9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:33.510219 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-9fff84fb-c264-4abd-b632-61800aa3ca9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:33.614448 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f07e03d-b0ef-4a88-ada1-76ee83b0bf2e admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:33.632598 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aa1bc2-c474-41bb-8137-5474c12ac40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:33.633520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa1bc2-c474-41bb-8137-5474c12ac40a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:33.636149 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa1bc2-c474-41bb-8137-5474c12ac40a admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:33.636560 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa1bc2-c474-41bb-8137-5474c12ac40a admin admin] PATH_INFO: `/v3/users/62bd2ed5d8f34a429d1a78d4498647a4` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:33.640008 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aa1bc2-c474-41bb-8137-5474c12ac40a admin admin] RBAC: Authorizing `identity:delete_user(user_id=62bd2ed5d8f34a429d1a78d4498647a4)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:33.646521 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aa1bc2-c474-41bb-8137-5474c12ac40a admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:33.705122 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 834/1670] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:33 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:33.720851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fff84fb-c264-4abd-b632-61800aa3ca9e None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:33.724163 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-9fff84fb-c264-4abd-b632-61800aa3ca9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:33.749765 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-98aa1bc2-c474-41bb-8137-5474c12ac40a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62bd2ed5d8f34a429d1a78d4498647a4'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:33.769038 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-98aa1bc2-c474-41bb-8137-5474c12ac40a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62bd2ed5d8f34a429d1a78d4498647a4'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:33.783801 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-98aa1bc2-c474-41bb-8137-5474c12ac40a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62bd2ed5d8f34a429d1a78d4498647a4'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:33.794940 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-98aa1bc2-c474-41bb-8137-5474c12ac40a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62bd2ed5d8f34a429d1a78d4498647a4'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:33.796194 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-98aa1bc2-c474-41bb-8137-5474c12ac40a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62bd2ed5d8f34a429d1a78d4498647a4'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:33.816147 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 837/1671] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:56:33 2022] DELETE /identity/v3/users/62bd2ed5d8f34a429d1a78d4498647a4 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:56:33.820621 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dababd6a-d71e-4ef0-8848-0830fd56318e None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:33.821267 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dababd6a-d71e-4ef0-8848-0830fd56318e None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:33.821740 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dababd6a-d71e-4ef0-8848-0830fd56318e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:33.868348 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-dababd6a-d71e-4ef0-8848-0830fd56318e None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:33.868877 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fff84fb-c264-4abd-b632-61800aa3ca9e None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:33.871199 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-dababd6a-d71e-4ef0-8848-0830fd56318e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:33.989298 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:34.003556 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee8bd98-3003-41dd-b189-f4625330039a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:34.004231 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8bd98-3003-41dd-b189-f4625330039a service cinder] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:34.004517 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8bd98-3003-41dd-b189-f4625330039a service cinder] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:34.004792 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8bd98-3003-41dd-b189-f4625330039a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:34.018674 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-8ee8bd98-3003-41dd-b189-f4625330039a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:34.028821 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:34.121160 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 838/1672] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 01:56:34.124005 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dababd6a-d71e-4ef0-8848-0830fd56318e None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:34.125888 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-dababd6a-d71e-4ef0-8848-0830fd56318e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:34.142823 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dababd6a-d71e-4ef0-8848-0830fd56318e None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:34.150051 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e822a-23a2-473d-99fd-13b578949f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:34.151560 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e822a-23a2-473d-99fd-13b578949f30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:34.151988 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e822a-23a2-473d-99fd-13b578949f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:34.170338 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e822a-23a2-473d-99fd-13b578949f30 admin admin] PATH_INFO: `/v3/projects/5cf82afb2ad54bb49446a1dd59f19c63` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:34.177636 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59e822a-23a2-473d-99fd-13b578949f30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cf82afb2ad54bb49446a1dd59f19c63)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:34.179724 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: WARNING py.warnings [None req-d59e822a-23a2-473d-99fd-13b578949f30 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 01:56:34.179724 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: warnings.warn(msg) Jun 23 01:56:34.179724 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: Jun 23 01:56:34.181828 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59e822a-23a2-473d-99fd-13b578949f30 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:34.242008 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee8bd98-3003-41dd-b189-f4625330039a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:34.243974 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee8bd98-3003-41dd-b189-f4625330039a service cinder] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:34.273770 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 835/1673] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:56:33 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 01:56:34.274488 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee8bd98-3003-41dd-b189-f4625330039a service cinder] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:34.283574 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-8ee8bd98-3003-41dd-b189-f4625330039a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:34.362509 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-d59e822a-23a2-473d-99fd-13b578949f30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cf82afb2ad54bb49446a1dd59f19c63', 'request_id': 'req-d59e822a-23a2-473d-99fd-13b578949f30'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:34.427602 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-d59e822a-23a2-473d-99fd-13b578949f30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cf82afb2ad54bb49446a1dd59f19c63', 'request_id': 'req-d59e822a-23a2-473d-99fd-13b578949f30'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:34.455498 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-d59e822a-23a2-473d-99fd-13b578949f30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cf82afb2ad54bb49446a1dd59f19c63', 'request_id': 'req-d59e822a-23a2-473d-99fd-13b578949f30'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:34.458579 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee8bd98-3003-41dd-b189-f4625330039a service cinder] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:34.490292 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-d59e822a-23a2-473d-99fd-13b578949f30 admin admin] The token cache is being invalidate because project 5cf82afb2ad54bb49446a1dd59f19c63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:56:34.492875 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-d59e822a-23a2-473d-99fd-13b578949f30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:34.494148 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-d59e822a-23a2-473d-99fd-13b578949f30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:34.499464 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 839/1674] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:56:34 2022] DELETE /identity/v3/projects/5cf82afb2ad54bb49446a1dd59f19c63 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:56:34.503646 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e822a-23a2-473d-99fd-13b578949f30 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:34.505911 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-d59e822a-23a2-473d-99fd-13b578949f30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:34.593086 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778dff1f-ac38-4f62-a5af-9fc285cb481f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:34.594219 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778dff1f-ac38-4f62-a5af-9fc285cb481f service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:34.594524 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778dff1f-ac38-4f62-a5af-9fc285cb481f service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:34.594925 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778dff1f-ac38-4f62-a5af-9fc285cb481f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:34.597795 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-778dff1f-ac38-4f62-a5af-9fc285cb481f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:34.685471 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e822a-23a2-473d-99fd-13b578949f30 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:34.715522 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b11acfb-c125-4bfa-87c5-7ca9e69203e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:34.716509 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11acfb-c125-4bfa-87c5-7ca9e69203e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:34.718225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11acfb-c125-4bfa-87c5-7ca9e69203e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:34.718225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11acfb-c125-4bfa-87c5-7ca9e69203e3 admin admin] PATH_INFO: `/v3/users/5537726541594995b03903ca768d8acc` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:34.725384 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b11acfb-c125-4bfa-87c5-7ca9e69203e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5537726541594995b03903ca768d8acc)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:34.730646 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b11acfb-c125-4bfa-87c5-7ca9e69203e3 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:34.777041 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778dff1f-ac38-4f62-a5af-9fc285cb481f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:34.779293 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778dff1f-ac38-4f62-a5af-9fc285cb481f service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:34.793495 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 836/1675] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:56:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:56:34.795242 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778dff1f-ac38-4f62-a5af-9fc285cb481f service neutron] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:34.797229 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-778dff1f-ac38-4f62-a5af-9fc285cb481f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:34.805376 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:34.869276 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-1b11acfb-c125-4bfa-87c5-7ca9e69203e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5537726541594995b03903ca768d8acc'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:34.890115 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-1b11acfb-c125-4bfa-87c5-7ca9e69203e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5537726541594995b03903ca768d8acc'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:34.894263 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:34.897519 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778dff1f-ac38-4f62-a5af-9fc285cb481f service neutron] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:34.907239 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-1b11acfb-c125-4bfa-87c5-7ca9e69203e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5537726541594995b03903ca768d8acc'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:34.910190 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79821fe-4793-450a-8a7e-fe6b404b08c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:34.913541 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79821fe-4793-450a-8a7e-fe6b404b08c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:34.913980 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79821fe-4793-450a-8a7e-fe6b404b08c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:34.914372 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79821fe-4793-450a-8a7e-fe6b404b08c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:34.918305 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-e79821fe-4793-450a-8a7e-fe6b404b08c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:34.922745 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-1b11acfb-c125-4bfa-87c5-7ca9e69203e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5537726541594995b03903ca768d8acc'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:34.924032 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-1b11acfb-c125-4bfa-87c5-7ca9e69203e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5537726541594995b03903ca768d8acc'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:34.953147 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 840/1676] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:56:34 2022] DELETE /identity/v3/users/5537726541594995b03903ca768d8acc => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:56:35.101337 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79821fe-4793-450a-8a7e-fe6b404b08c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:35.107503 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79821fe-4793-450a-8a7e-fe6b404b08c7 service cinder] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:35.144499 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 837/1677] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:56:34 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:56:35.952556 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b11acfb-c125-4bfa-87c5-7ca9e69203e3 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:35.954362 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-1b11acfb-c125-4bfa-87c5-7ca9e69203e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:36.098311 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b11acfb-c125-4bfa-87c5-7ca9e69203e3 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:36.109823 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c3419-7615-40df-b75d-8af85900f462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:36.110727 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c3419-7615-40df-b75d-8af85900f462 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:36.111012 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c3419-7615-40df-b75d-8af85900f462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:36.111314 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c3419-7615-40df-b75d-8af85900f462 admin admin] PATH_INFO: `/v3/projects/ed1b99fee624403eac4d4d016a86ba68` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:36.115245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209c3419-7615-40df-b75d-8af85900f462 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed1b99fee624403eac4d4d016a86ba68)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:36.117614 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209c3419-7615-40df-b75d-8af85900f462 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:36.218420 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-209c3419-7615-40df-b75d-8af85900f462 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed1b99fee624403eac4d4d016a86ba68', 'request_id': 'req-209c3419-7615-40df-b75d-8af85900f462'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:36.231720 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-209c3419-7615-40df-b75d-8af85900f462 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed1b99fee624403eac4d4d016a86ba68', 'request_id': 'req-209c3419-7615-40df-b75d-8af85900f462'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:36.232437 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3deaa-f84d-4a79-8d1a-ee6ca1a1a3ba None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:36.232897 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3deaa-f84d-4a79-8d1a-ee6ca1a1a3ba None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:36.233673 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3deaa-f84d-4a79-8d1a-ee6ca1a1a3ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:36.251102 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-209c3419-7615-40df-b75d-8af85900f462 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed1b99fee624403eac4d4d016a86ba68', 'request_id': 'req-209c3419-7615-40df-b75d-8af85900f462'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:36.263611 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-209c3419-7615-40df-b75d-8af85900f462 admin admin] The token cache is being invalidate because project ed1b99fee624403eac4d4d016a86ba68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:56:36.264230 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-209c3419-7615-40df-b75d-8af85900f462 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:36.265845 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-209c3419-7615-40df-b75d-8af85900f462 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:36.268493 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 841/1678] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:56:35 2022] DELETE /identity/v3/projects/ed1b99fee624403eac4d4d016a86ba68 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:56:36.274359 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ab61a-89d3-44b9-a2ce-c98f5fa0b09c None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:36.274840 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ab61a-89d3-44b9-a2ce-c98f5fa0b09c None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:36.276181 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ab61a-89d3-44b9-a2ce-c98f5fa0b09c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:36.295325 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-f5b3deaa-f84d-4a79-8d1a-ee6ca1a1a3ba None None] MFA Rules not processed for user `0f92d1dfc6c74b51aec7212f9a244a0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:36.297755 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-f5b3deaa-f84d-4a79-8d1a-ee6ca1a1a3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:36.302569 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-cc8ab61a-89d3-44b9-a2ce-c98f5fa0b09c None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:36.304090 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-cc8ab61a-89d3-44b9-a2ce-c98f5fa0b09c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:36.437282 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 842/1679] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:36.439038 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58843b01-a67e-4611-8ebe-ec10b86b44fd None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:36.439356 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58843b01-a67e-4611-8ebe-ec10b86b44fd None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:36.439641 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58843b01-a67e-4611-8ebe-ec10b86b44fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:36.473807 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 838/1680] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:36.483042 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-58843b01-a67e-4611-8ebe-ec10b86b44fd None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:36.485521 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b43a4ca-99a1-41e9-a9ce-8503248de0b2 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:36.485806 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b43a4ca-99a1-41e9-a9ce-8503248de0b2 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:36.486085 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b43a4ca-99a1-41e9-a9ce-8503248de0b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:36.488840 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-58843b01-a67e-4611-8ebe-ec10b86b44fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:36.518850 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 843/1681] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 01:56:36.523345 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58843b01-a67e-4611-8ebe-ec10b86b44fd None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:36.533299 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-6b43a4ca-99a1-41e9-a9ce-8503248de0b2 None None] MFA Rules not processed for user `0f92d1dfc6c74b51aec7212f9a244a0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:36.536888 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-6b43a4ca-99a1-41e9-a9ce-8503248de0b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:36.548122 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58843b01-a67e-4611-8ebe-ec10b86b44fd None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:36.561554 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 839/1682] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:36.569586 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075689f-ce8e-4308-8ad3-8e4eea13c1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:36.570271 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075689f-ce8e-4308-8ad3-8e4eea13c1b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:36.570560 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075689f-ce8e-4308-8ad3-8e4eea13c1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:36.570816 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075689f-ce8e-4308-8ad3-8e4eea13c1b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:36.571604 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0075689f-ce8e-4308-8ad3-8e4eea13c1b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:56:36.571911 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0075689f-ce8e-4308-8ad3-8e4eea13c1b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:36.574178 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b43a4ca-99a1-41e9-a9ce-8503248de0b2 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:36.575746 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-6b43a4ca-99a1-41e9-a9ce-8503248de0b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:36.577911 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0075689f-ce8e-4308-8ad3-8e4eea13c1b1 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:36.584999 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:36.626738 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 844/1683] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 01:56:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:56:36.631314 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075689f-ce8e-4308-8ad3-8e4eea13c1b1 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:36.657608 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075689f-ce8e-4308-8ad3-8e4eea13c1b1 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:36.668370 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac9d33-b43b-4f24-ba13-8c38f41f8f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:36.669047 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac9d33-b43b-4f24-ba13-8c38f41f8f83 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:36.669394 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac9d33-b43b-4f24-ba13-8c38f41f8f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:36.669676 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac9d33-b43b-4f24-ba13-8c38f41f8f83 admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:36.670371 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ac9d33-b43b-4f24-ba13-8c38f41f8f83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:36.672341 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ac9d33-b43b-4f24-ba13-8c38f41f8f83 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:36.696447 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:36.732759 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 845/1684] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:56:36 2022] POST /identity/v3/projects => generated 373 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:56:36.737891 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac9d33-b43b-4f24-ba13-8c38f41f8f83 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:36.754472 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b43a4ca-99a1-41e9-a9ce-8503248de0b2 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:36.891475 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac9d33-b43b-4f24-ba13-8c38f41f8f83 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:36.903973 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d641e0-e915-4b7e-a3bd-ea39e808fc79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:36.904672 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d641e0-e915-4b7e-a3bd-ea39e808fc79 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:36.904956 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d641e0-e915-4b7e-a3bd-ea39e808fc79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:36.905234 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d641e0-e915-4b7e-a3bd-ea39e808fc79 admin admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:36.906014 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d641e0-e915-4b7e-a3bd-ea39e808fc79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:36.908022 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d641e0-e915-4b7e-a3bd-ea39e808fc79 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:36.929445 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f0b9d-61f5-4968-ae7a-2bb7ff4d8134 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4e83df33b64a3ebf28f54e2d94fe63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:36.930171 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f0b9d-61f5-4968-ae7a-2bb7ff4d8134 service nova] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:36.930452 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f0b9d-61f5-4968-ae7a-2bb7ff4d8134 service nova] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:36.930730 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f0b9d-61f5-4968-ae7a-2bb7ff4d8134 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:36.959192 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 846/1685] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:56:36 2022] POST /identity/v3/users => generated 287 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 01:56:36.959192 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d641e0-e915-4b7e-a3bd-ea39e808fc79 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:36.966967 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-d2d641e0-e915-4b7e-a3bd-ea39e808fc79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:36.990139 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d641e0-e915-4b7e-a3bd-ea39e808fc79 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:37.003032 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe66ede9-6aec-4133-a163-ea39b02d2c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:37.003801 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe66ede9-6aec-4133-a163-ea39b02d2c0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:37.004084 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe66ede9-6aec-4133-a163-ea39b02d2c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:37.004356 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe66ede9-6aec-4133-a163-ea39b02d2c0a admin admin] PATH_INFO: `/v3/users/dd524429299f4964b12612bc0659bcd9` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:37.011273 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe66ede9-6aec-4133-a163-ea39b02d2c0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd524429299f4964b12612bc0659bcd9)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:37.018433 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe66ede9-6aec-4133-a163-ea39b02d2c0a admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:37.092569 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f0b9d-61f5-4968-ae7a-2bb7ff4d8134 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:37.094761 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f0b9d-61f5-4968-ae7a-2bb7ff4d8134 service nova] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:37.119390 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 840/1686] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:56:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:56:37.120356 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f0b9d-61f5-4968-ae7a-2bb7ff4d8134 service nova] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:37.159285 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f0b9d-61f5-4968-ae7a-2bb7ff4d8134 service nova] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:37.164090 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-fe66ede9-6aec-4133-a163-ea39b02d2c0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd524429299f4964b12612bc0659bcd9'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:37.190537 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad829e88-5744-4d7d-9e09-c6ef75274122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:37.191276 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad829e88-5744-4d7d-9e09-c6ef75274122 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:37.191560 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad829e88-5744-4d7d-9e09-c6ef75274122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:37.191841 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad829e88-5744-4d7d-9e09-c6ef75274122 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:37.192576 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad829e88-5744-4d7d-9e09-c6ef75274122 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:37.194361 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad829e88-5744-4d7d-9e09-c6ef75274122 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:37.205707 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-fe66ede9-6aec-4133-a163-ea39b02d2c0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd524429299f4964b12612bc0659bcd9'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:37.224203 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-fe66ede9-6aec-4133-a163-ea39b02d2c0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd524429299f4964b12612bc0659bcd9'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:37.235926 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 841/1687] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:56:37.236873 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad829e88-5744-4d7d-9e09-c6ef75274122 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:37.244093 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-ad829e88-5744-4d7d-9e09-c6ef75274122 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:37.246580 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-fe66ede9-6aec-4133-a163-ea39b02d2c0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd524429299f4964b12612bc0659bcd9'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:37.247353 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-fe66ede9-6aec-4133-a163-ea39b02d2c0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd524429299f4964b12612bc0659bcd9'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:37.260589 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 847/1688] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:56:36 2022] DELETE /identity/v3/users/dd524429299f4964b12612bc0659bcd9 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:56:37.261452 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe66ede9-6aec-4133-a163-ea39b02d2c0a admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:37.268179 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-fe66ede9-6aec-4133-a163-ea39b02d2c0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:37.286242 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad829e88-5744-4d7d-9e09-c6ef75274122 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:37.378424 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:37.445148 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe66ede9-6aec-4133-a163-ea39b02d2c0a admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:37.457885 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a182c1-7417-44ff-8716-7bbcbbc5cc6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:37.458266 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a182c1-7417-44ff-8716-7bbcbbc5cc6c admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:37.458550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a182c1-7417-44ff-8716-7bbcbbc5cc6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:37.458830 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a182c1-7417-44ff-8716-7bbcbbc5cc6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:37.459684 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a182c1-7417-44ff-8716-7bbcbbc5cc6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:37.461416 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a182c1-7417-44ff-8716-7bbcbbc5cc6c admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:37.481444 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 848/1689] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:56:37.482147 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad496bc-f71b-4b22-b71d-ab3a957de984 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:37.482436 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad496bc-f71b-4b22-b71d-ab3a957de984 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:37.482722 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad496bc-f71b-4b22-b71d-ab3a957de984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:37.528084 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5661e98-f15b-4880-aaf3-04f41d4963a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:37.529002 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5661e98-f15b-4880-aaf3-04f41d4963a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:37.529347 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5661e98-f15b-4880-aaf3-04f41d4963a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:37.529627 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5661e98-f15b-4880-aaf3-04f41d4963a8 admin admin] PATH_INFO: `/v3/projects/f87ecb9409d54151a1265e99e0cf6f0b` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:37.531305 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-bad496bc-f71b-4b22-b71d-ab3a957de984 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:37.532009 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5661e98-f15b-4880-aaf3-04f41d4963a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f87ecb9409d54151a1265e99e0cf6f0b)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:37.532622 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-bad496bc-f71b-4b22-b71d-ab3a957de984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:37.537000 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5661e98-f15b-4880-aaf3-04f41d4963a8 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:37.557367 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 849/1690] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:37.557367 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad496bc-f71b-4b22-b71d-ab3a957de984 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:37.558705 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:37.583530 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad496bc-f71b-4b22-b71d-ab3a957de984 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:37.597692 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dff3c61-9add-46ae-adf3-76f95effdf8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:37.598731 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff3c61-9add-46ae-adf3-76f95effdf8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:37.599013 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff3c61-9add-46ae-adf3-76f95effdf8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:37.599313 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff3c61-9add-46ae-adf3-76f95effdf8e admin admin] PATH_INFO: `/v3/projects/bdc94dc8b88f43f4a485c8b7cb4c03a4/users/e7439a877947465891932a5421ec28d5/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:37.647958 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dff3c61-9add-46ae-adf3-76f95effdf8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdc94dc8b88f43f4a485c8b7cb4c03a4, user_id=e7439a877947465891932a5421ec28d5, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:37.649907 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-b5661e98-f15b-4880-aaf3-04f41d4963a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f87ecb9409d54151a1265e99e0cf6f0b', 'request_id': 'req-b5661e98-f15b-4880-aaf3-04f41d4963a8'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:37.650667 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dff3c61-9add-46ae-adf3-76f95effdf8e admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:37.668395 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-b5661e98-f15b-4880-aaf3-04f41d4963a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f87ecb9409d54151a1265e99e0cf6f0b', 'request_id': 'req-b5661e98-f15b-4880-aaf3-04f41d4963a8'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:37.697073 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 850/1691] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:56:37 2022] PUT /identity/v3/projects/bdc94dc8b88f43f4a485c8b7cb4c03a4/users/e7439a877947465891932a5421ec28d5/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:56:37.697073 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dff3c61-9add-46ae-adf3-76f95effdf8e admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:37.701030 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-3dff3c61-9add-46ae-adf3-76f95effdf8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:37.708662 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-b5661e98-f15b-4880-aaf3-04f41d4963a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f87ecb9409d54151a1265e99e0cf6f0b', 'request_id': 'req-b5661e98-f15b-4880-aaf3-04f41d4963a8'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:37.770271 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-b5661e98-f15b-4880-aaf3-04f41d4963a8 admin admin] The token cache is being invalidate because project f87ecb9409d54151a1265e99e0cf6f0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:56:37.774098 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-b5661e98-f15b-4880-aaf3-04f41d4963a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:37.775695 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-b5661e98-f15b-4880-aaf3-04f41d4963a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:37.785123 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 842/1692] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:56:37 2022] DELETE /identity/v3/projects/f87ecb9409d54151a1265e99e0cf6f0b => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:56:37.793737 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777c6b2-b9fb-4125-a1db-4d22b98a6049 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:37.794926 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777c6b2-b9fb-4125-a1db-4d22b98a6049 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:37.795981 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777c6b2-b9fb-4125-a1db-4d22b98a6049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:37.850370 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dff3c61-9add-46ae-adf3-76f95effdf8e admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:37.861688 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0efadd-7a54-46a6-b31d-10ed30dd6430 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:37.862522 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0efadd-7a54-46a6-b31d-10ed30dd6430 service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:37.862956 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0efadd-7a54-46a6-b31d-10ed30dd6430 service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:37.863421 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0efadd-7a54-46a6-b31d-10ed30dd6430 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:37.872043 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-3a0efadd-7a54-46a6-b31d-10ed30dd6430 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:37.911159 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-2777c6b2-b9fb-4125-a1db-4d22b98a6049 None None] MFA Rules not processed for user `e7439a877947465891932a5421ec28d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:37.915260 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-2777c6b2-b9fb-4125-a1db-4d22b98a6049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:38.035555 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 843/1693] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:38.039344 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790af4a-f5c6-48bc-952f-c191bf3962fa None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:38.039344 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790af4a-f5c6-48bc-952f-c191bf3962fa None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:38.039717 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790af4a-f5c6-48bc-952f-c191bf3962fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:38.076851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-e790af4a-f5c6-48bc-952f-c191bf3962fa None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:38.076851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-e790af4a-f5c6-48bc-952f-c191bf3962fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:38.107144 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0efadd-7a54-46a6-b31d-10ed30dd6430 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:38.108643 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0efadd-7a54-46a6-b31d-10ed30dd6430 service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:38.137236 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 851/1694] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:56:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:56:38.137866 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0efadd-7a54-46a6-b31d-10ed30dd6430 service neutron] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:38.141323 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-3a0efadd-7a54-46a6-b31d-10ed30dd6430 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:38.168142 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0efadd-7a54-46a6-b31d-10ed30dd6430 service neutron] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:38.179032 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2b5670-670d-47b9-9a66-4bb86dfd2e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:38.179965 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b5670-670d-47b9-9a66-4bb86dfd2e4e admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:38.180401 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b5670-670d-47b9-9a66-4bb86dfd2e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:38.180819 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b5670-670d-47b9-9a66-4bb86dfd2e4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:38.181672 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2b5670-670d-47b9-9a66-4bb86dfd2e4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:38.186864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2b5670-670d-47b9-9a66-4bb86dfd2e4e admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:38.233649 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 844/1695] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:38.235416 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578fe5e6-e687-4e94-b370-ac69839e7400 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:38.235820 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578fe5e6-e687-4e94-b370-ac69839e7400 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:38.236207 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578fe5e6-e687-4e94-b370-ac69839e7400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:38.240280 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 852/1696] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:56:38 2022] POST /identity/v3/projects => generated 371 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:56:38.241982 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11119f82-8adc-4778-828a-111fb3959369 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:38.242381 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11119f82-8adc-4778-828a-111fb3959369 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:38.242768 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11119f82-8adc-4778-828a-111fb3959369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:38.283876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-11119f82-8adc-4778-828a-111fb3959369 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:38.285359 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-11119f82-8adc-4778-828a-111fb3959369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:38.298922 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-578fe5e6-e687-4e94-b370-ac69839e7400 None None] MFA Rules not processed for user `0287bd1260d24e5a94c00367b23c7a84`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:38.300574 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-578fe5e6-e687-4e94-b370-ac69839e7400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:38.327788 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:38.329807 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:38.394246 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 853/1697] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:38.395574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11119f82-8adc-4778-828a-111fb3959369 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:38.413104 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11119f82-8adc-4778-828a-111fb3959369 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:38.421442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2639318-1e8e-4e51-b943-83d2c8e4e64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:38.421595 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2639318-1e8e-4e51-b943-83d2c8e4e64f admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:38.422266 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2639318-1e8e-4e51-b943-83d2c8e4e64f admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:38.422550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2639318-1e8e-4e51-b943-83d2c8e4e64f admin admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:38.423568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2639318-1e8e-4e51-b943-83d2c8e4e64f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:38.426372 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2639318-1e8e-4e51-b943-83d2c8e4e64f admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:38.445447 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 845/1698] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:38.448282 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743fea21-e52d-4ba7-8b49-27b2c93175a9 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:38.448560 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743fea21-e52d-4ba7-8b49-27b2c93175a9 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:38.448851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743fea21-e52d-4ba7-8b49-27b2c93175a9 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:38.450643 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 846/1699] 10.222.0.26 () {54 vars in 912 bytes} [Thu Jun 23 01:56:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:56:38.451489 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743fea21-e52d-4ba7-8b49-27b2c93175a9 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:38.473539 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743fea21-e52d-4ba7-8b49-27b2c93175a9 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:38.479607 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 854/1700] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:56:38 2022] POST /identity/v3/users => generated 292 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:56:38.481835 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df96e974-eaa9-4cbb-94a2-3a4f89f03ba5 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:38.482135 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df96e974-eaa9-4cbb-94a2-3a4f89f03ba5 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:38.490858 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df96e974-eaa9-4cbb-94a2-3a4f89f03ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:38.491371 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63657e33-7bec-4530-8404-0a41795dbb4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:38.492184 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63657e33-7bec-4530-8404-0a41795dbb4c admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:38.494827 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63657e33-7bec-4530-8404-0a41795dbb4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:38.495238 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63657e33-7bec-4530-8404-0a41795dbb4c admin admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:38.496129 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63657e33-7bec-4530-8404-0a41795dbb4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:56:38.497361 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63657e33-7bec-4530-8404-0a41795dbb4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:38.500213 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63657e33-7bec-4530-8404-0a41795dbb4c admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:38.541361 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 847/1701] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 01:56:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:56:38.543994 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63657e33-7bec-4530-8404-0a41795dbb4c admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:38.548536 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-63657e33-7bec-4530-8404-0a41795dbb4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:38.558445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-df96e974-eaa9-4cbb-94a2-3a4f89f03ba5 None None] MFA Rules not processed for user `948e4905de44473bb7a45e1b521cf095`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:38.563063 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-df96e974-eaa9-4cbb-94a2-3a4f89f03ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:38.710041 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 855/1702] 10.222.0.26 () {58 vars in 1009 bytes} [Thu Jun 23 01:56:38 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:38.711531 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df96e974-eaa9-4cbb-94a2-3a4f89f03ba5 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:38.730728 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df96e974-eaa9-4cbb-94a2-3a4f89f03ba5 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:38.737539 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab45d4ac-9beb-4e5c-85a6-1d86f3ad357e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:38.738536 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab45d4ac-9beb-4e5c-85a6-1d86f3ad357e admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:38.739199 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab45d4ac-9beb-4e5c-85a6-1d86f3ad357e admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:38.739698 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab45d4ac-9beb-4e5c-85a6-1d86f3ad357e admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:38.741062 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab45d4ac-9beb-4e5c-85a6-1d86f3ad357e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:38.744796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab45d4ac-9beb-4e5c-85a6-1d86f3ad357e admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:38.760350 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63657e33-7bec-4530-8404-0a41795dbb4c admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:38.768412 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 856/1703] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:56:38.769643 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab45d4ac-9beb-4e5c-85a6-1d86f3ad357e admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:38.775057 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd31fc7-b869-4e2f-b46f-ab64a45ed564 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4e83df33b64a3ebf28f54e2d94fe63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:38.775773 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd31fc7-b869-4e2f-b46f-ab64a45ed564 service nova] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:38.776240 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd31fc7-b869-4e2f-b46f-ab64a45ed564 service nova] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:38.776525 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd31fc7-b869-4e2f-b46f-ab64a45ed564 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:38.796492 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd31fc7-b869-4e2f-b46f-ab64a45ed564 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:38.798141 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd31fc7-b869-4e2f-b46f-ab64a45ed564 service nova] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:38.814907 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab45d4ac-9beb-4e5c-85a6-1d86f3ad357e admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:38.822649 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f5cbd1-239f-4968-afda-fa08df02c172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:38.823242 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 848/1704] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:56:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:56:38.823678 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f5cbd1-239f-4968-afda-fa08df02c172 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:38.823966 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f5cbd1-239f-4968-afda-fa08df02c172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:38.824485 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f5cbd1-239f-4968-afda-fa08df02c172 admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:38.825267 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f5cbd1-239f-4968-afda-fa08df02c172 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:38.825460 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd31fc7-b869-4e2f-b46f-ab64a45ed564 service nova] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:38.827549 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f5cbd1-239f-4968-afda-fa08df02c172 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:38.830757 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-ffd31fc7-b869-4e2f-b46f-ab64a45ed564 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:38.855038 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd31fc7-b869-4e2f-b46f-ab64a45ed564 service nova] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:38.864157 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d830ea7-8e44-48cf-a195-c9f8f3af666d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:38.866082 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d830ea7-8e44-48cf-a195-c9f8f3af666d service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:38.866375 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d830ea7-8e44-48cf-a195-c9f8f3af666d service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:38.867235 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d830ea7-8e44-48cf-a195-c9f8f3af666d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:38.893646 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d830ea7-8e44-48cf-a195-c9f8f3af666d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:38.896276 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d830ea7-8e44-48cf-a195-c9f8f3af666d service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:38.909313 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 857/1705] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:56:38 2022] POST /identity/v3/projects => generated 371 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:56:38.911469 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f5cbd1-239f-4968-afda-fa08df02c172 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:38.915793 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 849/1706] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:56:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:56:38.918194 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d830ea7-8e44-48cf-a195-c9f8f3af666d service neutron] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:38.944680 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d830ea7-8e44-48cf-a195-c9f8f3af666d service neutron] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:39.114386 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f5cbd1-239f-4968-afda-fa08df02c172 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:39.121681 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e44dcf-330c-46c1-89ad-ccf3a6e2db06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:39.124631 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e44dcf-330c-46c1-89ad-ccf3a6e2db06 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:39.124918 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e44dcf-330c-46c1-89ad-ccf3a6e2db06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:39.125200 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e44dcf-330c-46c1-89ad-ccf3a6e2db06 admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:39.135958 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e44dcf-330c-46c1-89ad-ccf3a6e2db06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:39.140891 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e44dcf-330c-46c1-89ad-ccf3a6e2db06 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:39.143916 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0184e7-6ad7-4f49-9744-58e2e11925ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:39.144830 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0184e7-6ad7-4f49-9744-58e2e11925ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:39.145122 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0184e7-6ad7-4f49-9744-58e2e11925ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:39.145426 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0184e7-6ad7-4f49-9744-58e2e11925ed admin admin] PATH_INFO: `/v3/projects/0f06ad5367d54d33b4785ec1e493b31b/users/c9f0223dd5cf4458b7a4025b8928af66/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:39.204839 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0184e7-6ad7-4f49-9744-58e2e11925ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f06ad5367d54d33b4785ec1e493b31b, user_id=c9f0223dd5cf4458b7a4025b8928af66, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:39.207662 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0184e7-6ad7-4f49-9744-58e2e11925ed admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:39.220744 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 850/1707] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:56:38 2022] POST /identity/v3/users => generated 286 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:56:39.224680 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e44dcf-330c-46c1-89ad-ccf3a6e2db06 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:39.229532 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:39.261794 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 858/1708] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:56:38 2022] PUT /identity/v3/projects/0f06ad5367d54d33b4785ec1e493b31b/users/c9f0223dd5cf4458b7a4025b8928af66/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:56:39.268684 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:39.270342 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0184e7-6ad7-4f49-9744-58e2e11925ed admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:39.278390 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e44dcf-330c-46c1-89ad-ccf3a6e2db06 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:39.465105 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0184e7-6ad7-4f49-9744-58e2e11925ed admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:39.490280 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7821090a-5cd3-4407-817e-1453b85ccf0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:39.491032 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7821090a-5cd3-4407-817e-1453b85ccf0a admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:39.491383 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7821090a-5cd3-4407-817e-1453b85ccf0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:39.491851 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7821090a-5cd3-4407-817e-1453b85ccf0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:39.492612 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7821090a-5cd3-4407-817e-1453b85ccf0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:39.494255 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7821090a-5cd3-4407-817e-1453b85ccf0a admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:39.546337 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac3e589-f142-4358-91b3-e7218f83f538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:39.547304 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3e589-f142-4358-91b3-e7218f83f538 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:39.548919 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3e589-f142-4358-91b3-e7218f83f538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:39.550369 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3e589-f142-4358-91b3-e7218f83f538 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:39.555602 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 859/1709] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:56:39.555795 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac3e589-f142-4358-91b3-e7218f83f538 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:39.557341 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7821090a-5cd3-4407-817e-1453b85ccf0a admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:39.559497 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac3e589-f142-4358-91b3-e7218f83f538 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:39.568837 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7821090a-5cd3-4407-817e-1453b85ccf0a admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:39.575792 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42897f45-8d77-4570-8acf-971201b59389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:39.577980 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42897f45-8d77-4570-8acf-971201b59389 admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:39.578367 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42897f45-8d77-4570-8acf-971201b59389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:39.578634 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42897f45-8d77-4570-8acf-971201b59389 admin admin] PATH_INFO: `/v3/projects/0f06ad5367d54d33b4785ec1e493b31b/users/c9f0223dd5cf4458b7a4025b8928af66/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:39.589832 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42897f45-8d77-4570-8acf-971201b59389 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f06ad5367d54d33b4785ec1e493b31b, user_id=c9f0223dd5cf4458b7a4025b8928af66, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:39.593355 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42897f45-8d77-4570-8acf-971201b59389 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:39.596931 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 851/1710] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:56:39.597732 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac3e589-f142-4358-91b3-e7218f83f538 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:39.617468 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 860/1711] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:56:39 2022] PUT /identity/v3/projects/0f06ad5367d54d33b4785ec1e493b31b/users/c9f0223dd5cf4458b7a4025b8928af66/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 01:56:39.620638 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac3e589-f142-4358-91b3-e7218f83f538 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:39.632240 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f406e2be-d9dd-4f3c-9112-73564f920c11 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:39.632840 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f406e2be-d9dd-4f3c-9112-73564f920c11 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:39.633301 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f406e2be-d9dd-4f3c-9112-73564f920c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:39.710135 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-f406e2be-d9dd-4f3c-9112-73564f920c11 None None] MFA Rules not processed for user `c9f0223dd5cf4458b7a4025b8928af66`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:39.711998 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-f406e2be-d9dd-4f3c-9112-73564f920c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:39.825241 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738478b7-a5b5-4b0d-aaa9-a62dceca8540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:39.826007 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738478b7-a5b5-4b0d-aaa9-a62dceca8540 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:39.826239 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738478b7-a5b5-4b0d-aaa9-a62dceca8540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:39.826519 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738478b7-a5b5-4b0d-aaa9-a62dceca8540 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:39.827292 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738478b7-a5b5-4b0d-aaa9-a62dceca8540 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:39.828943 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738478b7-a5b5-4b0d-aaa9-a62dceca8540 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:39.852916 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 852/1712] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:56:39.853852 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738478b7-a5b5-4b0d-aaa9-a62dceca8540 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:39.872345 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738478b7-a5b5-4b0d-aaa9-a62dceca8540 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:39.884079 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b691d-f55f-4fa0-a82d-66da0f52b027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:39.884982 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b691d-f55f-4fa0-a82d-66da0f52b027 admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:39.885272 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b691d-f55f-4fa0-a82d-66da0f52b027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:39.885598 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b691d-f55f-4fa0-a82d-66da0f52b027 admin admin] PATH_INFO: `/v3/projects/637de42e242f405b9dfca71febe432fd/users/eb1b2d59e8134a67a1f5cea23990a22d/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:39.937922 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525b691d-f55f-4fa0-a82d-66da0f52b027 admin admin] RBAC: Authorizing `identity:create_grant(project_id=637de42e242f405b9dfca71febe432fd, user_id=eb1b2d59e8134a67a1f5cea23990a22d, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:39.942015 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525b691d-f55f-4fa0-a82d-66da0f52b027 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:39.971836 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 861/1714] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:39 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:39.973255 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f406e2be-d9dd-4f3c-9112-73564f920c11 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:39.978659 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 853/1714] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:56:39 2022] PUT /identity/v3/projects/637de42e242f405b9dfca71febe432fd/users/eb1b2d59e8134a67a1f5cea23990a22d/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:56:39.980299 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919e936-8442-42e0-b8ea-d1355a7dea6b None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:39.980770 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919e936-8442-42e0-b8ea-d1355a7dea6b None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:39.981062 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919e936-8442-42e0-b8ea-d1355a7dea6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:39.983455 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-f406e2be-d9dd-4f3c-9112-73564f920c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:40.001775 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f406e2be-d9dd-4f3c-9112-73564f920c11 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:40.118413 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60a7854-19c5-4c01-921f-2d24fff215cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:40.119357 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a7854-19c5-4c01-921f-2d24fff215cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:40.119694 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a7854-19c5-4c01-921f-2d24fff215cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:40.119910 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a7854-19c5-4c01-921f-2d24fff215cb admin admin] PATH_INFO: `/v3/projects/538de7f15e7341f69d9695c7dc9005d3` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:40.123664 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60a7854-19c5-4c01-921f-2d24fff215cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=538de7f15e7341f69d9695c7dc9005d3)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:40.125896 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60a7854-19c5-4c01-921f-2d24fff215cb admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:40.147451 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:40.286591 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-e60a7854-19c5-4c01-921f-2d24fff215cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '538de7f15e7341f69d9695c7dc9005d3', 'request_id': 'req-e60a7854-19c5-4c01-921f-2d24fff215cb'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:40.295726 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-b919e936-8442-42e0-b8ea-d1355a7dea6b None None] MFA Rules not processed for user `eb1b2d59e8134a67a1f5cea23990a22d`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:40.296929 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-b919e936-8442-42e0-b8ea-d1355a7dea6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:40.306223 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-e60a7854-19c5-4c01-921f-2d24fff215cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '538de7f15e7341f69d9695c7dc9005d3', 'request_id': 'req-e60a7854-19c5-4c01-921f-2d24fff215cb'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:40.335326 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-e60a7854-19c5-4c01-921f-2d24fff215cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '538de7f15e7341f69d9695c7dc9005d3', 'request_id': 'req-e60a7854-19c5-4c01-921f-2d24fff215cb'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:40.362352 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-e60a7854-19c5-4c01-921f-2d24fff215cb admin admin] The token cache is being invalidate because project 538de7f15e7341f69d9695c7dc9005d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:56:40.362669 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-e60a7854-19c5-4c01-921f-2d24fff215cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:40.363579 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-e60a7854-19c5-4c01-921f-2d24fff215cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:40.366311 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 862/1715] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:56:39 2022] DELETE /identity/v3/projects/538de7f15e7341f69d9695c7dc9005d3 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:56:40.397053 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60a7854-19c5-4c01-921f-2d24fff215cb admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:40.405113 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-e60a7854-19c5-4c01-921f-2d24fff215cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:40.599260 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 854/1716] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 617 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:40.607200 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60a7854-19c5-4c01-921f-2d24fff215cb admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:40.608882 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa77563-f921-41e2-b261-36bd21fc533f None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:40.609176 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa77563-f921-41e2-b261-36bd21fc533f None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:40.609456 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa77563-f921-41e2-b261-36bd21fc533f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:40.624813 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63045907-dec9-46b4-bf26-bfa71d3d2eea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:40.625423 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63045907-dec9-46b4-bf26-bfa71d3d2eea service cinder] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:40.626695 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63045907-dec9-46b4-bf26-bfa71d3d2eea service cinder] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:40.627121 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63045907-dec9-46b4-bf26-bfa71d3d2eea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:40.637309 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-63045907-dec9-46b4-bf26-bfa71d3d2eea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:40.639204 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:40.663443 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-7fa77563-f921-41e2-b261-36bd21fc533f None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:40.667264 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-7fa77563-f921-41e2-b261-36bd21fc533f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:40.829680 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63045907-dec9-46b4-bf26-bfa71d3d2eea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:40.837201 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63045907-dec9-46b4-bf26-bfa71d3d2eea service cinder] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:40.854884 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 855/1717] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:40.855846 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa77563-f921-41e2-b261-36bd21fc533f None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:40.863539 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-7fa77563-f921-41e2-b261-36bd21fc533f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:40.882622 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 863/1718] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:56:40 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:56:40.884101 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f0c19-9633-47ec-bd1e-c1dd0356cc37 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:40.884396 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f0c19-9633-47ec-bd1e-c1dd0356cc37 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:40.889728 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f0c19-9633-47ec-bd1e-c1dd0356cc37 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:40.896968 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 864/1719] 10.222.0.26 () {54 vars in 960 bytes} [Thu Jun 23 01:56:40 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:56:40.898527 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73742f7c-60fd-446b-b46b-5e4b2d015146 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:40.898735 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73742f7c-60fd-446b-b46b-5e4b2d015146 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:40.899018 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73742f7c-60fd-446b-b46b-5e4b2d015146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:40.974296 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-73742f7c-60fd-446b-b46b-5e4b2d015146 None None] MFA Rules not processed for user `0f92d1dfc6c74b51aec7212f9a244a0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:40.984536 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-73742f7c-60fd-446b-b46b-5e4b2d015146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:41.117349 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa77563-f921-41e2-b261-36bd21fc533f None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:41.130372 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:41.143590 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e5df53-d5f9-4749-ba61-50948107fbf6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4e83df33b64a3ebf28f54e2d94fe63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:41.145647 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5df53-d5f9-4749-ba61-50948107fbf6 service nova] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:41.147135 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5df53-d5f9-4749-ba61-50948107fbf6 service nova] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:41.147469 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5df53-d5f9-4749-ba61-50948107fbf6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:41.168318 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e5df53-d5f9-4749-ba61-50948107fbf6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:41.171300 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e5df53-d5f9-4749-ba61-50948107fbf6 service nova] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:41.182526 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 865/1720] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:41.185903 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a5295-7e19-4b09-aa3c-31f4aa1a952e None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:41.199962 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a5295-7e19-4b09-aa3c-31f4aa1a952e None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:41.202110 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a5295-7e19-4b09-aa3c-31f4aa1a952e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:41.238367 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 856/1721] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:56:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:56:41.246030 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e6606-8b24-43a3-bd35-a404db4a96b5 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:41.246859 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e6606-8b24-43a3-bd35-a404db4a96b5 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:41.247148 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e6606-8b24-43a3-bd35-a404db4a96b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:41.295493 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-345a5295-7e19-4b09-aa3c-31f4aa1a952e None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:41.298748 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-345a5295-7e19-4b09-aa3c-31f4aa1a952e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:41.316110 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-633e6606-8b24-43a3-bd35-a404db4a96b5 None None] MFA Rules not processed for user `e5bf3baee1b843fa89b65e5485f6bebb`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:41.322009 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-633e6606-8b24-43a3-bd35-a404db4a96b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:41.334434 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 866/1722] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:41.335424 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a5295-7e19-4b09-aa3c-31f4aa1a952e None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:41.368340 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a5295-7e19-4b09-aa3c-31f4aa1a952e None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:41.387711 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fb4ffb-9f3a-46b6-b1ac-0a2b407db55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:41.388662 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb4ffb-9f3a-46b6-b1ac-0a2b407db55a admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:41.389095 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb4ffb-9f3a-46b6-b1ac-0a2b407db55a admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:41.390202 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb4ffb-9f3a-46b6-b1ac-0a2b407db55a admin admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:41.391671 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fb4ffb-9f3a-46b6-b1ac-0a2b407db55a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:56:41.392145 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fb4ffb-9f3a-46b6-b1ac-0a2b407db55a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:41.396922 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fb4ffb-9f3a-46b6-b1ac-0a2b407db55a admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:41.433659 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 867/1723] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 01:56:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:56:41.435163 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fb4ffb-9f3a-46b6-b1ac-0a2b407db55a admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:41.469770 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fb4ffb-9f3a-46b6-b1ac-0a2b407db55a admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:41.483186 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f097b52-bdc4-4a32-80de-a4e4769983bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:41.484037 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f097b52-bdc4-4a32-80de-a4e4769983bf admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:41.484328 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f097b52-bdc4-4a32-80de-a4e4769983bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:41.485630 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f097b52-bdc4-4a32-80de-a4e4769983bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:41.486343 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f097b52-bdc4-4a32-80de-a4e4769983bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:41.492071 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f097b52-bdc4-4a32-80de-a4e4769983bf admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:41.512095 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 857/1724] 10.222.0.26 () {58 vars in 1057 bytes} [Thu Jun 23 01:56:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:41.521719 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633e6606-8b24-43a3-bd35-a404db4a96b5 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:41.543196 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633e6606-8b24-43a3-bd35-a404db4a96b5 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:41.566481 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 868/1725] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:56:41 2022] POST /identity/v3/projects => generated 365 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:56:41.568243 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f097b52-bdc4-4a32-80de-a4e4769983bf admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:41.569264 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74536c2b-e06d-461a-b7d1-30349a5f274a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5bf3baee1b843fa89b65e5485f6bebb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:41.573270 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74536c2b-e06d-461a-b7d1-30349a5f274a service placement] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:41.573270 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74536c2b-e06d-461a-b7d1-30349a5f274a service placement] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:41.573270 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74536c2b-e06d-461a-b7d1-30349a5f274a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:41.578692 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-74536c2b-e06d-461a-b7d1-30349a5f274a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:41.620462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74536c2b-e06d-461a-b7d1-30349a5f274a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:41.629130 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74536c2b-e06d-461a-b7d1-30349a5f274a service placement] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:41.666680 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 858/1726] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:56:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:56:41.853209 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f097b52-bdc4-4a32-80de-a4e4769983bf admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:41.872562 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9211a3cb-e319-49e0-9225-1636fe7ac65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:41.877355 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9211a3cb-e319-49e0-9225-1636fe7ac65a admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:41.878053 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9211a3cb-e319-49e0-9225-1636fe7ac65a admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:41.882632 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9211a3cb-e319-49e0-9225-1636fe7ac65a admin admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:41.885568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9211a3cb-e319-49e0-9225-1636fe7ac65a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:41.893559 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9211a3cb-e319-49e0-9225-1636fe7ac65a admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:41.944745 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde9dbb-fb63-4a72-b690-5494b5f399c1 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:41.945121 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde9dbb-fb63-4a72-b690-5494b5f399c1 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:41.945566 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde9dbb-fb63-4a72-b690-5494b5f399c1 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:41.948125 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 859/1727] 10.222.0.26 () {54 vars in 912 bytes} [Thu Jun 23 01:56:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 01:56:41.974558 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 869/1728] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:56:41 2022] POST /identity/v3/users => generated 283 bytes in 405 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:56:41.975952 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde9dbb-fb63-4a72-b690-5494b5f399c1 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:41.993694 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c58441-5872-48e4-aabd-c6d580aaed12 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:41.995541 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c58441-5872-48e4-aabd-c6d580aaed12 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:41.995970 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c58441-5872-48e4-aabd-c6d580aaed12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:42.003705 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde9dbb-fb63-4a72-b690-5494b5f399c1 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:42.026487 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad3e162-f5cc-4aff-b3e3-e51c2129b7f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:42.028669 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad3e162-f5cc-4aff-b3e3-e51c2129b7f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:42.028871 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad3e162-f5cc-4aff-b3e3-e51c2129b7f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:42.035639 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad3e162-f5cc-4aff-b3e3-e51c2129b7f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:42.036478 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad3e162-f5cc-4aff-b3e3-e51c2129b7f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:42.044525 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad3e162-f5cc-4aff-b3e3-e51c2129b7f9 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:42.046451 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-00c58441-5872-48e4-aabd-c6d580aaed12 None None] MFA Rules not processed for user `948e4905de44473bb7a45e1b521cf095`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:42.052272 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-00c58441-5872-48e4-aabd-c6d580aaed12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:42.060948 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:42.082044 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 860/1729] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:56:41 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:56:42.086011 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad3e162-f5cc-4aff-b3e3-e51c2129b7f9 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:42.115969 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad3e162-f5cc-4aff-b3e3-e51c2129b7f9 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:42.135474 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95605bc2-bd1d-45c9-b84a-b8480acddf94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:42.137020 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95605bc2-bd1d-45c9-b84a-b8480acddf94 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:42.137908 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95605bc2-bd1d-45c9-b84a-b8480acddf94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:42.138405 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95605bc2-bd1d-45c9-b84a-b8480acddf94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:42.139467 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95605bc2-bd1d-45c9-b84a-b8480acddf94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:42.144213 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95605bc2-bd1d-45c9-b84a-b8480acddf94 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:42.187407 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 861/1730] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:56:42.189799 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95605bc2-bd1d-45c9-b84a-b8480acddf94 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:42.198725 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-95605bc2-bd1d-45c9-b84a-b8480acddf94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:42.254637 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95605bc2-bd1d-45c9-b84a-b8480acddf94 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:42.276592 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 870/1731] 10.222.0.26 () {58 vars in 1009 bytes} [Thu Jun 23 01:56:41 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:42.286693 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72296d7d-19a4-4114-85af-82b6d443e6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:42.287778 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72296d7d-19a4-4114-85af-82b6d443e6c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:42.288134 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72296d7d-19a4-4114-85af-82b6d443e6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:42.288481 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72296d7d-19a4-4114-85af-82b6d443e6c8 admin admin] PATH_INFO: `/v3/users/e7439a877947465891932a5421ec28d5` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:42.288610 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c58441-5872-48e4-aabd-c6d580aaed12 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:42.290350 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72296d7d-19a4-4114-85af-82b6d443e6c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7439a877947465891932a5421ec28d5)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:42.292557 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72296d7d-19a4-4114-85af-82b6d443e6c8 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:42.333996 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c58441-5872-48e4-aabd-c6d580aaed12 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:42.348142 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361d377c-811f-461d-ab43-cb3e29e384cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:42.350396 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d377c-811f-461d-ab43-cb3e29e384cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:42.350696 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d377c-811f-461d-ab43-cb3e29e384cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:42.351692 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d377c-811f-461d-ab43-cb3e29e384cc admin admin] PATH_INFO: `/v3/projects/9b296a3e9448401fb0c0a0f9ffab90af/users/3bdc5fb46a20446c8b1c9718b607971e/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:42.390208 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-72296d7d-19a4-4114-85af-82b6d443e6c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7439a877947465891932a5421ec28d5'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:42.408546 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361d377c-811f-461d-ab43-cb3e29e384cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b296a3e9448401fb0c0a0f9ffab90af, user_id=3bdc5fb46a20446c8b1c9718b607971e, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:42.417383 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361d377c-811f-461d-ab43-cb3e29e384cc admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:42.421599 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-72296d7d-19a4-4114-85af-82b6d443e6c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7439a877947465891932a5421ec28d5'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:42.444242 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-72296d7d-19a4-4114-85af-82b6d443e6c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7439a877947465891932a5421ec28d5'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:42.448914 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 871/1732] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:56:42 2022] PUT /identity/v3/projects/9b296a3e9448401fb0c0a0f9ffab90af/users/3bdc5fb46a20446c8b1c9718b607971e/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:56:42.450480 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361d377c-811f-461d-ab43-cb3e29e384cc admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:42.462193 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-361d377c-811f-461d-ab43-cb3e29e384cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:42.465537 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-72296d7d-19a4-4114-85af-82b6d443e6c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7439a877947465891932a5421ec28d5'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:42.466528 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-72296d7d-19a4-4114-85af-82b6d443e6c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7439a877947465891932a5421ec28d5'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:42.479800 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 862/1733] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:56:42 2022] DELETE /identity/v3/users/e7439a877947465891932a5421ec28d5 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:56:42.483909 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324cf182-38ac-4ba0-b12d-a4e5c721c558 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:42.484374 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324cf182-38ac-4ba0-b12d-a4e5c721c558 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:42.485214 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324cf182-38ac-4ba0-b12d-a4e5c721c558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:42.548803 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:42.555372 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-324cf182-38ac-4ba0-b12d-a4e5c721c558 None None] MFA Rules not processed for user `3bdc5fb46a20446c8b1c9718b607971e`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:42.560942 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-324cf182-38ac-4ba0-b12d-a4e5c721c558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:42.627028 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361d377c-811f-461d-ab43-cb3e29e384cc admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:42.726891 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 863/1734] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:42 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:42.728248 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f864370d-3a90-4f50-ab46-a2f76f852b38 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:42.728549 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f864370d-3a90-4f50-ab46-a2f76f852b38 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:42.728830 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f864370d-3a90-4f50-ab46-a2f76f852b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:42.784439 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-f864370d-3a90-4f50-ab46-a2f76f852b38 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:42.786926 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-f864370d-3a90-4f50-ab46-a2f76f852b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:42.826507 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9004e93e-c162-4d27-bc5a-63c9c4e287e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:42.827385 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9004e93e-c162-4d27-bc5a-63c9c4e287e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:42.827746 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9004e93e-c162-4d27-bc5a-63c9c4e287e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:42.828346 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9004e93e-c162-4d27-bc5a-63c9c4e287e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:42.831974 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-9004e93e-c162-4d27-bc5a-63c9c4e287e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:42.869664 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9004e93e-c162-4d27-bc5a-63c9c4e287e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:42.871353 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9004e93e-c162-4d27-bc5a-63c9c4e287e8 service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:42.893667 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:42.901658 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 872/1735] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:56:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:56:42.903293 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9004e93e-c162-4d27-bc5a-63c9c4e287e8 service neutron] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:42.906659 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-9004e93e-c162-4d27-bc5a-63c9c4e287e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:42.927905 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 864/1736] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:42.936265 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9004e93e-c162-4d27-bc5a-63c9c4e287e8 service neutron] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:42.954280 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0245fd2a-690c-4ebd-8320-c504dae24303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:42.955174 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0245fd2a-690c-4ebd-8320-c504dae24303 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:42.956322 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0245fd2a-690c-4ebd-8320-c504dae24303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:42.958301 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0245fd2a-690c-4ebd-8320-c504dae24303 admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:42.959929 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0245fd2a-690c-4ebd-8320-c504dae24303 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:42.964967 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0245fd2a-690c-4ebd-8320-c504dae24303 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:42.974278 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f864370d-3a90-4f50-ab46-a2f76f852b38 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:42.975949 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-f864370d-3a90-4f50-ab46-a2f76f852b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:42.999509 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f864370d-3a90-4f50-ab46-a2f76f852b38 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:43.016684 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1959333-5e0d-4dd7-87eb-046e4c904ca5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:43.017405 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1959333-5e0d-4dd7-87eb-046e4c904ca5 service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:43.017694 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1959333-5e0d-4dd7-87eb-046e4c904ca5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:43.017970 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1959333-5e0d-4dd7-87eb-046e4c904ca5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:43.041935 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 873/1737] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:56:42 2022] POST /identity/v3/projects => generated 365 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:56:43.045369 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0245fd2a-690c-4ebd-8320-c504dae24303 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:43.064171 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1959333-5e0d-4dd7-87eb-046e4c904ca5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:43.065658 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1959333-5e0d-4dd7-87eb-046e4c904ca5 service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:43.121015 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 865/1738] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:56:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 01:56:43.211687 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0245fd2a-690c-4ebd-8320-c504dae24303 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:43.224608 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f61b2-d8f5-4c21-ba65-4db82f4a888e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:43.227637 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f61b2-d8f5-4c21-ba65-4db82f4a888e admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:43.228796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f61b2-d8f5-4c21-ba65-4db82f4a888e admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:43.229105 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f61b2-d8f5-4c21-ba65-4db82f4a888e admin admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:43.230771 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062f61b2-d8f5-4c21-ba65-4db82f4a888e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:43.237187 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062f61b2-d8f5-4c21-ba65-4db82f4a888e admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:43.284719 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 874/1739] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:56:43 2022] POST /identity/v3/users => generated 289 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:56:43.286352 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1959333-5e0d-4dd7-87eb-046e4c904ca5 service neutron] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:43.320210 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1959333-5e0d-4dd7-87eb-046e4c904ca5 service neutron] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:43.343388 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df7e458-82cf-4590-8b3a-085f626e4c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:43.344121 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df7e458-82cf-4590-8b3a-085f626e4c06 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:43.344356 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df7e458-82cf-4590-8b3a-085f626e4c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:43.344641 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df7e458-82cf-4590-8b3a-085f626e4c06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:43.345404 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df7e458-82cf-4590-8b3a-085f626e4c06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:43.346985 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df7e458-82cf-4590-8b3a-085f626e4c06 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:43.376728 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 866/1740] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:56:43 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:56:43.378334 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f61b2-d8f5-4c21-ba65-4db82f4a888e admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:43.393018 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f61b2-d8f5-4c21-ba65-4db82f4a888e admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:43.419945 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab977fed-f808-496e-bd24-095dce8c35bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:43.422916 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab977fed-f808-496e-bd24-095dce8c35bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:43.427083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab977fed-f808-496e-bd24-095dce8c35bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:43.427645 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab977fed-f808-496e-bd24-095dce8c35bc admin admin] PATH_INFO: `/v3/projects/ce8d9cda737a4812a53fa2a528214f1b/users/0ab9af5719c64da9b2abe2aef2bb93d8/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:43.476680 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab977fed-f808-496e-bd24-095dce8c35bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce8d9cda737a4812a53fa2a528214f1b, user_id=0ab9af5719c64da9b2abe2aef2bb93d8, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:43.480193 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab977fed-f808-496e-bd24-095dce8c35bc admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:43.509818 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 875/1741] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:56:43 2022] PUT /identity/v3/projects/ce8d9cda737a4812a53fa2a528214f1b/users/0ab9af5719c64da9b2abe2aef2bb93d8/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:56:43.514127 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df7e458-82cf-4590-8b3a-085f626e4c06 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:43.529845 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df7e458-82cf-4590-8b3a-085f626e4c06 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:43.664499 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b0119f-504c-4e03-b2a5-16a3b696e822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:43.665215 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b0119f-504c-4e03-b2a5-16a3b696e822 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:43.666642 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b0119f-504c-4e03-b2a5-16a3b696e822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:43.667182 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b0119f-504c-4e03-b2a5-16a3b696e822 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:43.668523 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b0119f-504c-4e03-b2a5-16a3b696e822 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:43.670761 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b0119f-504c-4e03-b2a5-16a3b696e822 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:43.703312 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab977fed-f808-496e-bd24-095dce8c35bc admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:43.715673 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 867/1742] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:56:43 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:56:43.727453 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab977fed-f808-496e-bd24-095dce8c35bc admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:43.759727 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e682bf-8bd9-4e08-83c2-4c50c145708a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:43.760655 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e682bf-8bd9-4e08-83c2-4c50c145708a admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:43.760932 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e682bf-8bd9-4e08-83c2-4c50c145708a admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:43.770225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e682bf-8bd9-4e08-83c2-4c50c145708a admin admin] PATH_INFO: `/v3/projects/ce8d9cda737a4812a53fa2a528214f1b/users/0ab9af5719c64da9b2abe2aef2bb93d8/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:43.778465 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e682bf-8bd9-4e08-83c2-4c50c145708a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce8d9cda737a4812a53fa2a528214f1b, user_id=0ab9af5719c64da9b2abe2aef2bb93d8, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:43.788571 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e682bf-8bd9-4e08-83c2-4c50c145708a admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:43.824880 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc400a-2a34-44b7-8bcf-806d680eea46 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:43.825185 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc400a-2a34-44b7-8bcf-806d680eea46 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:43.825462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc400a-2a34-44b7-8bcf-806d680eea46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:43.866920 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 876/1743] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:56:43 2022] PUT /identity/v3/projects/ce8d9cda737a4812a53fa2a528214f1b/users/0ab9af5719c64da9b2abe2aef2bb93d8/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:56:43.883325 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4c00f-1df7-4e60-8a38-68bc74f056c8 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:43.885622 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4c00f-1df7-4e60-8a38-68bc74f056c8 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:43.891367 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4c00f-1df7-4e60-8a38-68bc74f056c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:43.962665 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:43.971704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-63b4c00f-1df7-4e60-8a38-68bc74f056c8 None None] MFA Rules not processed for user `0ab9af5719c64da9b2abe2aef2bb93d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:43.975372 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-63b4c00f-1df7-4e60-8a38-68bc74f056c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:44.053425 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-69cc400a-2a34-44b7-8bcf-806d680eea46 None None] MFA Rules not processed for user `ababea3e976f46889f2b1980edfee883`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:44.054706 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-69cc400a-2a34-44b7-8bcf-806d680eea46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:44.290035 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 877/1744] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:43 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 408 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:44.295763 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442623d6-5064-4b11-ae5b-642734aec539 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:44.300811 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442623d6-5064-4b11-ae5b-642734aec539 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:44.301097 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442623d6-5064-4b11-ae5b-642734aec539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:44.348155 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 868/1745] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 525 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 01:56:44.349630 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc400a-2a34-44b7-8bcf-806d680eea46 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:44.361673 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-69cc400a-2a34-44b7-8bcf-806d680eea46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:44.412670 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-442623d6-5064-4b11-ae5b-642734aec539 None None] MFA Rules not processed for user `56362ad5e50c4f4e8fb694f845d02924`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:44.413933 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-442623d6-5064-4b11-ae5b-642734aec539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:44.530001 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:44.567638 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc400a-2a34-44b7-8bcf-806d680eea46 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:44.581045 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba00617a-2608-4b89-bea0-97c7ba0a71e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:44.582047 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00617a-2608-4b89-bea0-97c7ba0a71e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:44.584693 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00617a-2608-4b89-bea0-97c7ba0a71e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:44.585377 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00617a-2608-4b89-bea0-97c7ba0a71e2 admin admin] PATH_INFO: `/v3/projects/bdc94dc8b88f43f4a485c8b7cb4c03a4` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:44.592061 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba00617a-2608-4b89-bea0-97c7ba0a71e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdc94dc8b88f43f4a485c8b7cb4c03a4)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:44.595681 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba00617a-2608-4b89-bea0-97c7ba0a71e2 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:44.613606 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 878/1746] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:44.614550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442623d6-5064-4b11-ae5b-642734aec539 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:44.631341 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-442623d6-5064-4b11-ae5b-642734aec539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:44.711767 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-ba00617a-2608-4b89-bea0-97c7ba0a71e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdc94dc8b88f43f4a485c8b7cb4c03a4', 'request_id': 'req-ba00617a-2608-4b89-bea0-97c7ba0a71e2'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:44.737726 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-ba00617a-2608-4b89-bea0-97c7ba0a71e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdc94dc8b88f43f4a485c8b7cb4c03a4', 'request_id': 'req-ba00617a-2608-4b89-bea0-97c7ba0a71e2'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:44.788153 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-ba00617a-2608-4b89-bea0-97c7ba0a71e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdc94dc8b88f43f4a485c8b7cb4c03a4', 'request_id': 'req-ba00617a-2608-4b89-bea0-97c7ba0a71e2'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:44.819621 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-ba00617a-2608-4b89-bea0-97c7ba0a71e2 admin admin] The token cache is being invalidate because project bdc94dc8b88f43f4a485c8b7cb4c03a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:56:44.821589 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-ba00617a-2608-4b89-bea0-97c7ba0a71e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:44.824003 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-ba00617a-2608-4b89-bea0-97c7ba0a71e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:44.827143 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 869/1747] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:56:44 2022] DELETE /identity/v3/projects/bdc94dc8b88f43f4a485c8b7cb4c03a4 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:56:44.829518 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba00617a-2608-4b89-bea0-97c7ba0a71e2 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:44.835150 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442623d6-5064-4b11-ae5b-642734aec539 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:44.835670 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-ba00617a-2608-4b89-bea0-97c7ba0a71e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:44.972110 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:44.991698 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fca4fe-7605-4508-a81b-b954a6d7b026 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:44.992375 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fca4fe-7605-4508-a81b-b954a6d7b026 service cinder] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:44.992661 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fca4fe-7605-4508-a81b-b954a6d7b026 service cinder] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:44.992938 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fca4fe-7605-4508-a81b-b954a6d7b026 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:45.007059 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-f0fca4fe-7605-4508-a81b-b954a6d7b026 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:45.121427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba00617a-2608-4b89-bea0-97c7ba0a71e2 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:45.181928 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d5e580-d248-4fc9-a052-26e98abe7c59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4e83df33b64a3ebf28f54e2d94fe63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:45.181928 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5e580-d248-4fc9-a052-26e98abe7c59 service nova] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:45.181928 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5e580-d248-4fc9-a052-26e98abe7c59 service nova] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:45.181928 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5e580-d248-4fc9-a052-26e98abe7c59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:45.191373 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-45d5e580-d248-4fc9-a052-26e98abe7c59 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:45.348614 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fca4fe-7605-4508-a81b-b954a6d7b026 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:45.350876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fca4fe-7605-4508-a81b-b954a6d7b026 service cinder] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:45.383394 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 879/1748] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:56:44 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 769 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:56:45.391025 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657aed0-7148-4d18-9386-3f6645632d9c None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:45.392405 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657aed0-7148-4d18-9386-3f6645632d9c None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:45.393130 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657aed0-7148-4d18-9386-3f6645632d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:45.442289 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-2657aed0-7148-4d18-9386-3f6645632d9c None None] MFA Rules not processed for user `56362ad5e50c4f4e8fb694f845d02924`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:45.444848 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-2657aed0-7148-4d18-9386-3f6645632d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:45.513715 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d5e580-d248-4fc9-a052-26e98abe7c59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:45.518676 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d5e580-d248-4fc9-a052-26e98abe7c59 service nova] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:45.551085 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 870/1749] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:56:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 722 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:56:45.552849 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d5e580-d248-4fc9-a052-26e98abe7c59 service nova] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:45.560579 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-45d5e580-d248-4fc9-a052-26e98abe7c59 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:45.590595 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 880/1750] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:45.598368 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda7aa7c-4c47-4892-965c-089871d0b976 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:45.598652 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda7aa7c-4c47-4892-965c-089871d0b976 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:45.598928 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda7aa7c-4c47-4892-965c-089871d0b976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:45.638129 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:45.660591 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-bda7aa7c-4c47-4892-965c-089871d0b976 None None] MFA Rules not processed for user `0287bd1260d24e5a94c00367b23c7a84`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:45.666014 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-bda7aa7c-4c47-4892-965c-089871d0b976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:45.815001 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d5e580-d248-4fc9-a052-26e98abe7c59 service nova] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:45.823127 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39484eb-07b3-43a6-b3fa-c9e61062ac79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:45.824265 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39484eb-07b3-43a6-b3fa-c9e61062ac79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:45.824547 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39484eb-07b3-43a6-b3fa-c9e61062ac79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:45.824844 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39484eb-07b3-43a6-b3fa-c9e61062ac79 admin admin] PATH_INFO: `/v3/users/c9f0223dd5cf4458b7a4025b8928af66` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:45.826808 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39484eb-07b3-43a6-b3fa-c9e61062ac79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9f0223dd5cf4458b7a4025b8928af66)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:45.829103 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39484eb-07b3-43a6-b3fa-c9e61062ac79 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:45.951548 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 881/1751] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:45.953369 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda7aa7c-4c47-4892-965c-089871d0b976 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:45.959133 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-bda7aa7c-4c47-4892-965c-089871d0b976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:46.023903 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-a39484eb-07b3-43a6-b3fa-c9e61062ac79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9f0223dd5cf4458b7a4025b8928af66'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:46.036807 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda7aa7c-4c47-4892-965c-089871d0b976 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:46.074379 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec37989c-c94f-4787-a20b-dd2e6d804986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:46.078624 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec37989c-c94f-4787-a20b-dd2e6d804986 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:46.078996 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec37989c-c94f-4787-a20b-dd2e6d804986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:46.079309 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec37989c-c94f-4787-a20b-dd2e6d804986 admin admin] PATH_INFO: `/v3/users/eb1b2d59e8134a67a1f5cea23990a22d` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:46.093884 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec37989c-c94f-4787-a20b-dd2e6d804986 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb1b2d59e8134a67a1f5cea23990a22d)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:46.096006 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec37989c-c94f-4787-a20b-dd2e6d804986 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:46.100431 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-a39484eb-07b3-43a6-b3fa-c9e61062ac79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9f0223dd5cf4458b7a4025b8928af66'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:46.140848 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-a39484eb-07b3-43a6-b3fa-c9e61062ac79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9f0223dd5cf4458b7a4025b8928af66'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:46.175313 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-a39484eb-07b3-43a6-b3fa-c9e61062ac79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9f0223dd5cf4458b7a4025b8928af66'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:46.180000 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-a39484eb-07b3-43a6-b3fa-c9e61062ac79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9f0223dd5cf4458b7a4025b8928af66'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:46.213803 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 871/1752] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:56:45 2022] DELETE /identity/v3/users/c9f0223dd5cf4458b7a4025b8928af66 => generated 0 bytes in 658 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:56:46.214482 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39484eb-07b3-43a6-b3fa-c9e61062ac79 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:46.239320 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-a39484eb-07b3-43a6-b3fa-c9e61062ac79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:46.327348 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:46.359862 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-ec37989c-c94f-4787-a20b-dd2e6d804986 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb1b2d59e8134a67a1f5cea23990a22d'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:46.378458 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-ec37989c-c94f-4787-a20b-dd2e6d804986 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb1b2d59e8134a67a1f5cea23990a22d'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:46.390658 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-ec37989c-c94f-4787-a20b-dd2e6d804986 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb1b2d59e8134a67a1f5cea23990a22d'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:46.414825 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-ec37989c-c94f-4787-a20b-dd2e6d804986 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb1b2d59e8134a67a1f5cea23990a22d'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:46.418062 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-ec37989c-c94f-4787-a20b-dd2e6d804986 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb1b2d59e8134a67a1f5cea23990a22d'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:46.439117 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 882/1753] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:56:45 2022] DELETE /identity/v3/users/eb1b2d59e8134a67a1f5cea23990a22d => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:56:46.444298 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb2f86b-de7d-4b71-9c8a-bba5837999ae None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:46.445849 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb2f86b-de7d-4b71-9c8a-bba5837999ae None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:46.446461 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb2f86b-de7d-4b71-9c8a-bba5837999ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:46.508293 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-ebb2f86b-de7d-4b71-9c8a-bba5837999ae None None] MFA Rules not processed for user `168e6181ad22460990f1f7361b483364`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:46.509542 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-ebb2f86b-de7d-4b71-9c8a-bba5837999ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:46.530987 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39484eb-07b3-43a6-b3fa-c9e61062ac79 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:46.687456 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:46.689876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 883/1754] 10.222.0.26 () {58 vars in 1009 bytes} [Thu Jun 23 01:56:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:46.691426 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7afa072-3fc4-4fb0-95d2-6a5314acad11 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:46.693850 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7afa072-3fc4-4fb0-95d2-6a5314acad11 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:46.694172 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7afa072-3fc4-4fb0-95d2-6a5314acad11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:46.769924 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-c7afa072-3fc4-4fb0-95d2-6a5314acad11 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:46.771195 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-c7afa072-3fc4-4fb0-95d2-6a5314acad11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:46.795624 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac831c7-3f7f-427c-8aae-c3c01423428d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4e83df33b64a3ebf28f54e2d94fe63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:46.796306 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac831c7-3f7f-427c-8aae-c3c01423428d service nova] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:46.796761 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac831c7-3f7f-427c-8aae-c3c01423428d service nova] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:46.797188 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac831c7-3f7f-427c-8aae-c3c01423428d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:46.799169 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-eac831c7-3f7f-427c-8aae-c3c01423428d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:46.910977 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac831c7-3f7f-427c-8aae-c3c01423428d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:46.917920 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac831c7-3f7f-427c-8aae-c3c01423428d service nova] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:46.937815 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 872/1755] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:56:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 724 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:56:46.938965 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac831c7-3f7f-427c-8aae-c3c01423428d service nova] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:46.941159 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-eac831c7-3f7f-427c-8aae-c3c01423428d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:47.003047 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 884/1756] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:47.035936 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac831c7-3f7f-427c-8aae-c3c01423428d service nova] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:47.041629 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff271b4-1944-407d-8770-5e87caabb412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:47.042564 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff271b4-1944-407d-8770-5e87caabb412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:47.042853 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff271b4-1944-407d-8770-5e87caabb412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:47.043333 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff271b4-1944-407d-8770-5e87caabb412 admin admin] PATH_INFO: `/v3/users/3bdc5fb46a20446c8b1c9718b607971e` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:47.055293 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff271b4-1944-407d-8770-5e87caabb412 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bdc5fb46a20446c8b1c9718b607971e)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:47.057761 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff271b4-1944-407d-8770-5e87caabb412 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:47.101711 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7afa072-3fc4-4fb0-95d2-6a5314acad11 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:47.103446 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-c7afa072-3fc4-4fb0-95d2-6a5314acad11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:47.185627 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-aff271b4-1944-407d-8770-5e87caabb412 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bdc5fb46a20446c8b1c9718b607971e'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:47.256096 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-aff271b4-1944-407d-8770-5e87caabb412 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bdc5fb46a20446c8b1c9718b607971e'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:47.292434 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-aff271b4-1944-407d-8770-5e87caabb412 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bdc5fb46a20446c8b1c9718b607971e'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:47.309733 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:47.311511 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:47.319401 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-aff271b4-1944-407d-8770-5e87caabb412 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bdc5fb46a20446c8b1c9718b607971e'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:47.322293 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-aff271b4-1944-407d-8770-5e87caabb412 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bdc5fb46a20446c8b1c9718b607971e'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:47.327842 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7afa072-3fc4-4fb0-95d2-6a5314acad11 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:47.346178 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cb8ced-c730-46f6-810d-bd417e10504e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28e59688485241c4b6d51f99bbb3d059', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:47.348035 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb8ced-c730-46f6-810d-bd417e10504e service swift] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:47.348487 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb8ced-c730-46f6-810d-bd417e10504e service swift] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:47.351945 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb8ced-c730-46f6-810d-bd417e10504e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:47.356992 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 873/1757] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:56:46 2022] DELETE /identity/v3/users/3bdc5fb46a20446c8b1c9718b607971e => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:56:47.360649 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff271b4-1944-407d-8770-5e87caabb412 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:47.361937 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-c8cb8ced-c730-46f6-810d-bd417e10504e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:47.368932 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-aff271b4-1944-407d-8770-5e87caabb412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:47.457063 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cb8ced-c730-46f6-810d-bd417e10504e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:47.461713 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cb8ced-c730-46f6-810d-bd417e10504e service swift] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:47.538225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 885/1758] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Jun 23 01:56:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 01:56:47.546848 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061a662-8e74-4ed0-9a34-ba5fe8e96aae None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:47.548642 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061a662-8e74-4ed0-9a34-ba5fe8e96aae None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:47.548956 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061a662-8e74-4ed0-9a34-ba5fe8e96aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:47.618334 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff271b4-1944-407d-8770-5e87caabb412 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:47.649049 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510fd1c3-3649-4d4a-8146-b84392a9dc6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:47.650340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510fd1c3-3649-4d4a-8146-b84392a9dc6f service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:47.650626 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510fd1c3-3649-4d4a-8146-b84392a9dc6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:47.653853 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510fd1c3-3649-4d4a-8146-b84392a9dc6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:47.667354 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-b061a662-8e74-4ed0-9a34-ba5fe8e96aae None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:47.673276 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-b061a662-8e74-4ed0-9a34-ba5fe8e96aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:47.676878 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-510fd1c3-3649-4d4a-8146-b84392a9dc6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:47.920580 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 886/1759] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:47.920580 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061a662-8e74-4ed0-9a34-ba5fe8e96aae None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:47.924138 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-b061a662-8e74-4ed0-9a34-ba5fe8e96aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:47.977615 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061a662-8e74-4ed0-9a34-ba5fe8e96aae None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:47.983340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510fd1c3-3649-4d4a-8146-b84392a9dc6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:47.996313 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a3e43-5a5f-4476-9210-8651f654cb52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:47.998957 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a3e43-5a5f-4476-9210-8651f654cb52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:48.000378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a3e43-5a5f-4476-9210-8651f654cb52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:48.002685 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a3e43-5a5f-4476-9210-8651f654cb52 admin admin] PATH_INFO: `/v3/projects/0f06ad5367d54d33b4785ec1e493b31b` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:48.004610 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510fd1c3-3649-4d4a-8146-b84392a9dc6f service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:48.005610 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a3e43-5a5f-4476-9210-8651f654cb52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f06ad5367d54d33b4785ec1e493b31b)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:48.011679 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a3e43-5a5f-4476-9210-8651f654cb52 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:48.085389 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 874/1760] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:56:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 728 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:56:48.095567 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510fd1c3-3649-4d4a-8146-b84392a9dc6f service neutron] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:48.136004 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510fd1c3-3649-4d4a-8146-b84392a9dc6f service neutron] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:48.149891 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c9c08-094a-4a67-9465-5c94d76de734 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:48.150897 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c9c08-094a-4a67-9465-5c94d76de734 service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:48.151597 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c9c08-094a-4a67-9465-5c94d76de734 service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:48.151885 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c9c08-094a-4a67-9465-5c94d76de734 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:48.187699 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4c9c08-094a-4a67-9465-5c94d76de734 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:48.188486 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4c9c08-094a-4a67-9465-5c94d76de734 service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:48.204301 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-be9a3e43-5a5f-4476-9210-8651f654cb52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f06ad5367d54d33b4785ec1e493b31b', 'request_id': 'req-be9a3e43-5a5f-4476-9210-8651f654cb52'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:48.227308 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-be9a3e43-5a5f-4476-9210-8651f654cb52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f06ad5367d54d33b4785ec1e493b31b', 'request_id': 'req-be9a3e43-5a5f-4476-9210-8651f654cb52'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:48.228170 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 875/1761] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:56:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:56:48.248720 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-be9a3e43-5a5f-4476-9210-8651f654cb52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f06ad5367d54d33b4785ec1e493b31b', 'request_id': 'req-be9a3e43-5a5f-4476-9210-8651f654cb52'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:48.271243 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-be9a3e43-5a5f-4476-9210-8651f654cb52 admin admin] The token cache is being invalidate because project 0f06ad5367d54d33b4785ec1e493b31b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:56:48.271452 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-be9a3e43-5a5f-4476-9210-8651f654cb52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:48.273224 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-be9a3e43-5a5f-4476-9210-8651f654cb52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:48.278655 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 887/1762] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:56:47 2022] DELETE /identity/v3/projects/0f06ad5367d54d33b4785ec1e493b31b => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:56:48.284424 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4637261a-d472-468f-bd21-ec60db4e34a3 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:48.284727 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4637261a-d472-468f-bd21-ec60db4e34a3 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:48.285056 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4637261a-d472-468f-bd21-ec60db4e34a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:48.354794 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-4637261a-d472-468f-bd21-ec60db4e34a3 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:48.360987 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-4637261a-d472-468f-bd21-ec60db4e34a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:48.608951 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 876/1763] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:48.610824 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f59cdd-0927-4e76-8a3e-5d490de036fe None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:48.611417 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f59cdd-0927-4e76-8a3e-5d490de036fe None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:48.611519 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f59cdd-0927-4e76-8a3e-5d490de036fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:48.657813 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-46f59cdd-0927-4e76-8a3e-5d490de036fe None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:48.659156 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-46f59cdd-0927-4e76-8a3e-5d490de036fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:48.739339 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 888/1764] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:48.741394 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4637261a-d472-468f-bd21-ec60db4e34a3 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:48.761877 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4637261a-d472-468f-bd21-ec60db4e34a3 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:48.769587 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460cd202-bbc7-4433-a1af-1a8563f428ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:48.770668 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460cd202-bbc7-4433-a1af-1a8563f428ec admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:48.771800 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460cd202-bbc7-4433-a1af-1a8563f428ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:48.772267 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460cd202-bbc7-4433-a1af-1a8563f428ec admin admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:48.774252 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460cd202-bbc7-4433-a1af-1a8563f428ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:56:48.775330 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460cd202-bbc7-4433-a1af-1a8563f428ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:48.784580 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460cd202-bbc7-4433-a1af-1a8563f428ec admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:48.821895 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 877/1765] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 01:56:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:56:48.826521 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f59cdd-0927-4e76-8a3e-5d490de036fe None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:48.857345 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f59cdd-0927-4e76-8a3e-5d490de036fe None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:48.890464 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf62c0fa-3567-49df-8346-219645ca0153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:48.891652 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf62c0fa-3567-49df-8346-219645ca0153 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:48.892212 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf62c0fa-3567-49df-8346-219645ca0153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:48.892773 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf62c0fa-3567-49df-8346-219645ca0153 admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:48.893688 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf62c0fa-3567-49df-8346-219645ca0153 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:48.896072 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf62c0fa-3567-49df-8346-219645ca0153 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:48.986099 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 889/1766] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:56:48 2022] POST /identity/v3/projects => generated 367 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:56:48.990356 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460cd202-bbc7-4433-a1af-1a8563f428ec admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:49.013032 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460cd202-bbc7-4433-a1af-1a8563f428ec admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:49.065877 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:49.227118 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a8295d-3933-4519-8faa-9cb7cca7b57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:49.227864 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8295d-3933-4519-8faa-9cb7cca7b57c admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:49.228146 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8295d-3933-4519-8faa-9cb7cca7b57c admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:49.228421 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8295d-3933-4519-8faa-9cb7cca7b57c admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:49.231969 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a8295d-3933-4519-8faa-9cb7cca7b57c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:49.234494 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a8295d-3933-4519-8faa-9cb7cca7b57c admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:49.312655 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 878/1767] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:56:48 2022] POST /identity/v3/users => generated 284 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:56:49.317920 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf62c0fa-3567-49df-8346-219645ca0153 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:49.362183 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf62c0fa-3567-49df-8346-219645ca0153 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:49.374149 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8ef69c-3421-4945-ab18-d0d16ffb26e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:49.375199 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ef69c-3421-4945-ab18-d0d16ffb26e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:49.375760 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ef69c-3421-4945-ab18-d0d16ffb26e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:49.376267 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ef69c-3421-4945-ab18-d0d16ffb26e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:49.377209 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8ef69c-3421-4945-ab18-d0d16ffb26e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:49.388497 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8ef69c-3421-4945-ab18-d0d16ffb26e5 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:49.435142 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 890/1768] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:56:49 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:56:49.439535 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a8295d-3933-4519-8faa-9cb7cca7b57c admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:49.459133 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a8295d-3933-4519-8faa-9cb7cca7b57c admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:49.478570 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8395ff63-2dca-41a1-aa34-50dbde531588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:49.480049 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8395ff63-2dca-41a1-aa34-50dbde531588 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:49.480313 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8395ff63-2dca-41a1-aa34-50dbde531588 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:49.482040 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8395ff63-2dca-41a1-aa34-50dbde531588 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:49.482040 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8395ff63-2dca-41a1-aa34-50dbde531588 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:49.485335 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8395ff63-2dca-41a1-aa34-50dbde531588 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:49.507059 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 879/1769] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:56:49 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:56:49.511241 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8ef69c-3421-4945-ab18-d0d16ffb26e5 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:49.543065 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8ef69c-3421-4945-ab18-d0d16ffb26e5 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:49.581054 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e346501e-d59e-4f41-a7bf-4c5677cb415c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:49.587033 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346501e-d59e-4f41-a7bf-4c5677cb415c admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:49.587597 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346501e-d59e-4f41-a7bf-4c5677cb415c admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:49.588055 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346501e-d59e-4f41-a7bf-4c5677cb415c admin admin] PATH_INFO: `/v3/projects/b63e012e8eaa4c91aa1d2cbac60f7490/users/cdeaf0128bee46b1a595d96bba6a2817/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:49.648252 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e346501e-d59e-4f41-a7bf-4c5677cb415c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b63e012e8eaa4c91aa1d2cbac60f7490, user_id=cdeaf0128bee46b1a595d96bba6a2817, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:49.673760 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e346501e-d59e-4f41-a7bf-4c5677cb415c admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:49.724686 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 891/1770] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:56:49 2022] PUT /identity/v3/projects/b63e012e8eaa4c91aa1d2cbac60f7490/users/cdeaf0128bee46b1a595d96bba6a2817/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:56:49.737716 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4c356-3a82-45cb-8d43-aa9c88a36819 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:49.738053 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4c356-3a82-45cb-8d43-aa9c88a36819 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:49.738488 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4c356-3a82-45cb-8d43-aa9c88a36819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:49.841819 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-1be4c356-3a82-45cb-8d43-aa9c88a36819 None None] MFA Rules not processed for user `cdeaf0128bee46b1a595d96bba6a2817`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:49.843041 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-1be4c356-3a82-45cb-8d43-aa9c88a36819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:50.067466 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 880/1771] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:49 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 01:56:50.072863 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e346501e-d59e-4f41-a7bf-4c5677cb415c admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:50.100654 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:50.305104 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e346501e-d59e-4f41-a7bf-4c5677cb415c admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:50.316008 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de72327d-932a-406e-89ea-cf8bed49e5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:50.317226 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de72327d-932a-406e-89ea-cf8bed49e5f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:50.317226 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de72327d-932a-406e-89ea-cf8bed49e5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:50.317411 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de72327d-932a-406e-89ea-cf8bed49e5f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:50.318833 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de72327d-932a-406e-89ea-cf8bed49e5f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:50.320163 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de72327d-932a-406e-89ea-cf8bed49e5f5 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:50.425237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 892/1772] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:56:50 2022] POST /identity/v3/projects => generated 369 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:56:50.427189 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4c356-3a82-45cb-8d43-aa9c88a36819 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:50.440479 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4c356-3a82-45cb-8d43-aa9c88a36819 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:50.724225 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca46dc-c059-4b66-b802-fe29d7d40f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:50.725192 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca46dc-c059-4b66-b802-fe29d7d40f6c admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:50.725326 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca46dc-c059-4b66-b802-fe29d7d40f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:50.725604 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca46dc-c059-4b66-b802-fe29d7d40f6c admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:50.726308 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ca46dc-c059-4b66-b802-fe29d7d40f6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:50.734296 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ca46dc-c059-4b66-b802-fe29d7d40f6c admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:50.813313 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de72327d-932a-406e-89ea-cf8bed49e5f5 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:50.830625 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 881/1773] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:56:50 2022] POST /identity/v3/users => generated 291 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:56:50.840394 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de72327d-932a-406e-89ea-cf8bed49e5f5 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:50.858034 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc28b8b-d026-4168-9833-bf8dc431e710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:50.860485 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc28b8b-d026-4168-9833-bf8dc431e710 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:50.861864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc28b8b-d026-4168-9833-bf8dc431e710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:50.862256 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc28b8b-d026-4168-9833-bf8dc431e710 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:50.864561 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc28b8b-d026-4168-9833-bf8dc431e710 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:50.868937 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc28b8b-d026-4168-9833-bf8dc431e710 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:50.917477 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 893/1774] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:56:50 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:56:50.919144 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca46dc-c059-4b66-b802-fe29d7d40f6c admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:50.943930 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca46dc-c059-4b66-b802-fe29d7d40f6c admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:50.957936 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22387d2e-934e-476e-b9db-7c7eec851bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:50.959601 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22387d2e-934e-476e-b9db-7c7eec851bd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:50.962084 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22387d2e-934e-476e-b9db-7c7eec851bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:50.962663 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22387d2e-934e-476e-b9db-7c7eec851bd3 admin admin] PATH_INFO: `/v3/projects/305f45bf58e84e8f90a9b99fcfa26477/users/d60b20faff2f4d658219096f3e3e4cb7/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:51.009949 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:51.016393 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22387d2e-934e-476e-b9db-7c7eec851bd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=305f45bf58e84e8f90a9b99fcfa26477, user_id=d60b20faff2f4d658219096f3e3e4cb7, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:51.019973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22387d2e-934e-476e-b9db-7c7eec851bd3 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:51.044531 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 882/1775] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:56:50 2022] PUT /identity/v3/projects/305f45bf58e84e8f90a9b99fcfa26477/users/d60b20faff2f4d658219096f3e3e4cb7/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:56:51.068868 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc28b8b-d026-4168-9833-bf8dc431e710 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:51.091076 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc28b8b-d026-4168-9833-bf8dc431e710 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:51.282804 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2fff7-a358-4926-8257-2fb804d2364e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:51.283573 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2fff7-a358-4926-8257-2fb804d2364e admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:51.283799 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2fff7-a358-4926-8257-2fb804d2364e admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:51.284076 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2fff7-a358-4926-8257-2fb804d2364e admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:51.289087 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f2fff7-a358-4926-8257-2fb804d2364e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:51.294005 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f2fff7-a358-4926-8257-2fb804d2364e admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:51.308989 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22387d2e-934e-476e-b9db-7c7eec851bd3 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:51.308989 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-22387d2e-934e-476e-b9db-7c7eec851bd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:51.329782 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22387d2e-934e-476e-b9db-7c7eec851bd3 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:51.351610 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 894/1776] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:56:51 2022] GET /identity/v3/roles => generated 1420 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:56:51.352964 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2fff7-a358-4926-8257-2fb804d2364e admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:51.355767 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9871e66-b408-4046-8143-1a7b538c46af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:51.362162 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9871e66-b408-4046-8143-1a7b538c46af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:51.362476 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9871e66-b408-4046-8143-1a7b538c46af admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:51.362760 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9871e66-b408-4046-8143-1a7b538c46af admin admin] PATH_INFO: `/v3/projects/9b296a3e9448401fb0c0a0f9ffab90af` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:51.364334 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9871e66-b408-4046-8143-1a7b538c46af admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b296a3e9448401fb0c0a0f9ffab90af)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:51.369502 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9871e66-b408-4046-8143-1a7b538c46af admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:51.381237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2fff7-a358-4926-8257-2fb804d2364e admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:51.395463 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde34843-979b-4164-8c30-7d3a6a5c7fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:51.397031 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde34843-979b-4164-8c30-7d3a6a5c7fc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:51.399916 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde34843-979b-4164-8c30-7d3a6a5c7fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:51.401409 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde34843-979b-4164-8c30-7d3a6a5c7fc4 admin admin] PATH_INFO: `/v3/projects/305f45bf58e84e8f90a9b99fcfa26477/users/d60b20faff2f4d658219096f3e3e4cb7/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:51.408099 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde34843-979b-4164-8c30-7d3a6a5c7fc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=305f45bf58e84e8f90a9b99fcfa26477, user_id=d60b20faff2f4d658219096f3e3e4cb7, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:51.416514 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde34843-979b-4164-8c30-7d3a6a5c7fc4 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:51.458821 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 895/1777] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:56:51 2022] PUT /identity/v3/projects/305f45bf58e84e8f90a9b99fcfa26477/users/d60b20faff2f4d658219096f3e3e4cb7/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:56:51.463245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b109b0-2f5b-44c5-901c-f2ff3d45a877 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:51.464234 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b109b0-2f5b-44c5-901c-f2ff3d45a877 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:51.466878 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b109b0-2f5b-44c5-901c-f2ff3d45a877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:51.531768 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-84b109b0-2f5b-44c5-901c-f2ff3d45a877 None None] MFA Rules not processed for user `d60b20faff2f4d658219096f3e3e4cb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:51.533358 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-84b109b0-2f5b-44c5-901c-f2ff3d45a877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:51.582524 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-e9871e66-b408-4046-8143-1a7b538c46af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b296a3e9448401fb0c0a0f9ffab90af', 'request_id': 'req-e9871e66-b408-4046-8143-1a7b538c46af'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:51.613717 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-e9871e66-b408-4046-8143-1a7b538c46af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b296a3e9448401fb0c0a0f9ffab90af', 'request_id': 'req-e9871e66-b408-4046-8143-1a7b538c46af'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:51.637598 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-e9871e66-b408-4046-8143-1a7b538c46af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b296a3e9448401fb0c0a0f9ffab90af', 'request_id': 'req-e9871e66-b408-4046-8143-1a7b538c46af'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:51.667996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-e9871e66-b408-4046-8143-1a7b538c46af admin admin] The token cache is being invalidate because project 9b296a3e9448401fb0c0a0f9ffab90af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:56:51.670969 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-e9871e66-b408-4046-8143-1a7b538c46af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:51.672883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-e9871e66-b408-4046-8143-1a7b538c46af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:51.678438 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 883/1778] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:56:51 2022] DELETE /identity/v3/projects/9b296a3e9448401fb0c0a0f9ffab90af => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:56:51.683360 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9871e66-b408-4046-8143-1a7b538c46af admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:51.686264 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-e9871e66-b408-4046-8143-1a7b538c46af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:51.823136 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:51.875732 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 896/1779] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:51 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 409 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:51.884425 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b109b0-2f5b-44c5-901c-f2ff3d45a877 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:51.886405 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-84b109b0-2f5b-44c5-901c-f2ff3d45a877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:52.016818 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9871e66-b408-4046-8143-1a7b538c46af admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:52.031191 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3095b74-4bcb-42f4-abe1-0845bd00c068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:52.033440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3095b74-4bcb-42f4-abe1-0845bd00c068 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:52.033725 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3095b74-4bcb-42f4-abe1-0845bd00c068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:52.034008 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3095b74-4bcb-42f4-abe1-0845bd00c068 admin admin] PATH_INFO: `/v3/users/0ab9af5719c64da9b2abe2aef2bb93d8` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:52.035959 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3095b74-4bcb-42f4-abe1-0845bd00c068 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ab9af5719c64da9b2abe2aef2bb93d8)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:52.044616 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3095b74-4bcb-42f4-abe1-0845bd00c068 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:52.123155 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b109b0-2f5b-44c5-901c-f2ff3d45a877 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:52.234751 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-f3095b74-4bcb-42f4-abe1-0845bd00c068 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ab9af5719c64da9b2abe2aef2bb93d8'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:52.285643 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-f3095b74-4bcb-42f4-abe1-0845bd00c068 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ab9af5719c64da9b2abe2aef2bb93d8'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:52.313064 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-f3095b74-4bcb-42f4-abe1-0845bd00c068 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ab9af5719c64da9b2abe2aef2bb93d8'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:52.324788 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=80,keystone:INSERT=3,keystone:DELETE=17 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:52.349590 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-f3095b74-4bcb-42f4-abe1-0845bd00c068 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ab9af5719c64da9b2abe2aef2bb93d8'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:52.351098 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-f3095b74-4bcb-42f4-abe1-0845bd00c068 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ab9af5719c64da9b2abe2aef2bb93d8'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:52.376977 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34872941-f6ec-4757-b5f1-11514f5e3fa6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:52.381862 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34872941-f6ec-4757-b5f1-11514f5e3fa6 service cinder] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:52.383342 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34872941-f6ec-4757-b5f1-11514f5e3fa6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:52.385624 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34872941-f6ec-4757-b5f1-11514f5e3fa6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:52.394034 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-34872941-f6ec-4757-b5f1-11514f5e3fa6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:52.395601 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 884/1780] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:56:51 2022] DELETE /identity/v3/users/0ab9af5719c64da9b2abe2aef2bb93d8 => generated 0 bytes in 713 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 01:56:52.664131 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34872941-f6ec-4757-b5f1-11514f5e3fa6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:52.665856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34872941-f6ec-4757-b5f1-11514f5e3fa6 service cinder] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:52.719629 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 897/1781] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:56:51 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 836 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:56:52.730105 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3095b74-4bcb-42f4-abe1-0845bd00c068 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:52.731598 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-f3095b74-4bcb-42f4-abe1-0845bd00c068 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:52.838117 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34872941-f6ec-4757-b5f1-11514f5e3fa6 service cinder] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:52.840015 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-34872941-f6ec-4757-b5f1-11514f5e3fa6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:52.864225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34872941-f6ec-4757-b5f1-11514f5e3fa6 service cinder] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:53.027425 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3095b74-4bcb-42f4-abe1-0845bd00c068 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:53.042891 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cafa03-9dab-4322-a95a-54355db7f6b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:53.049245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cafa03-9dab-4322-a95a-54355db7f6b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:53.052871 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cafa03-9dab-4322-a95a-54355db7f6b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:53.055468 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cafa03-9dab-4322-a95a-54355db7f6b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:53.061800 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce2773-9399-48b7-8bb4-ca004d58dc9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:53.062851 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-84cafa03-9dab-4322-a95a-54355db7f6b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:53.065918 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce2773-9399-48b7-8bb4-ca004d58dc9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:53.066213 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce2773-9399-48b7-8bb4-ca004d58dc9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:53.066493 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce2773-9399-48b7-8bb4-ca004d58dc9b admin admin] PATH_INFO: `/v3/projects/637de42e242f405b9dfca71febe432fd` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:53.075811 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ce2773-9399-48b7-8bb4-ca004d58dc9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=637de42e242f405b9dfca71febe432fd)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:53.083416 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ce2773-9399-48b7-8bb4-ca004d58dc9b admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:53.111319 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:53.230137 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-18ce2773-9399-48b7-8bb4-ca004d58dc9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '637de42e242f405b9dfca71febe432fd', 'request_id': 'req-18ce2773-9399-48b7-8bb4-ca004d58dc9b'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:53.247974 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-18ce2773-9399-48b7-8bb4-ca004d58dc9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '637de42e242f405b9dfca71febe432fd', 'request_id': 'req-18ce2773-9399-48b7-8bb4-ca004d58dc9b'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:53.288863 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-18ce2773-9399-48b7-8bb4-ca004d58dc9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '637de42e242f405b9dfca71febe432fd', 'request_id': 'req-18ce2773-9399-48b7-8bb4-ca004d58dc9b'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:53.310368 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cafa03-9dab-4322-a95a-54355db7f6b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:53.313676 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cafa03-9dab-4322-a95a-54355db7f6b0 service cinder] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:53.318881 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-18ce2773-9399-48b7-8bb4-ca004d58dc9b admin admin] The token cache is being invalidate because project 637de42e242f405b9dfca71febe432fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:56:53.319430 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-18ce2773-9399-48b7-8bb4-ca004d58dc9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:53.320503 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-18ce2773-9399-48b7-8bb4-ca004d58dc9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:53.330251 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 885/1782] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:56:52 2022] DELETE /identity/v3/projects/637de42e242f405b9dfca71febe432fd => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:56:53.335706 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028469a-3a56-44ee-a610-6fc9f47257eb None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:53.353817 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028469a-3a56-44ee-a610-6fc9f47257eb None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:53.354749 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028469a-3a56-44ee-a610-6fc9f47257eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:53.378449 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 898/1783] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:56:52 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 01:56:53.384479 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a62451-c376-4d11-800f-3b30ae213392 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:53.384771 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a62451-c376-4d11-800f-3b30ae213392 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:53.385054 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a62451-c376-4d11-800f-3b30ae213392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:53.438969 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-7028469a-3a56-44ee-a610-6fc9f47257eb None None] MFA Rules not processed for user `948e4905de44473bb7a45e1b521cf095`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:53.452611 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-7028469a-3a56-44ee-a610-6fc9f47257eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:53.474734 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-66a62451-c376-4d11-800f-3b30ae213392 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:53.476065 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-66a62451-c376-4d11-800f-3b30ae213392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:53.757410 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 886/1784] 10.222.0.16 () {58 vars in 1015 bytes} [Thu Jun 23 01:56:53 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 427 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:53.836746 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 899/1785] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 453 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:53.846175 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c665d42-b262-4057-ac9e-5169db1f0262 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:53.846518 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c665d42-b262-4057-ac9e-5169db1f0262 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:53.846762 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c665d42-b262-4057-ac9e-5169db1f0262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:53.901006 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-1c665d42-b262-4057-ac9e-5169db1f0262 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:53.902807 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-1c665d42-b262-4057-ac9e-5169db1f0262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:53.944323 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 887/1786] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:53.946504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a62451-c376-4d11-800f-3b30ae213392 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:53.968541 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a62451-c376-4d11-800f-3b30ae213392 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:53.987023 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ca23d-6d67-49f8-9182-6246464498b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:53.990602 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ca23d-6d67-49f8-9182-6246464498b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:53.991645 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ca23d-6d67-49f8-9182-6246464498b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:53.996678 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ca23d-6d67-49f8-9182-6246464498b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:53.997701 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ca23d-6d67-49f8-9182-6246464498b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:56:53.999559 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ca23d-6d67-49f8-9182-6246464498b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:54.001463 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ca23d-6d67-49f8-9182-6246464498b1 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:54.052820 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c665d42-b262-4057-ac9e-5169db1f0262 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:54.074503 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c665d42-b262-4057-ac9e-5169db1f0262 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:54.074872 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 900/1787] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 01:56:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:56:54.096934 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b9f2d1-a004-4fd1-884a-ff0fd3c8fe8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:54.097938 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b9f2d1-a004-4fd1-884a-ff0fd3c8fe8d admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:54.098462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b9f2d1-a004-4fd1-884a-ff0fd3c8fe8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:54.098810 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b9f2d1-a004-4fd1-884a-ff0fd3c8fe8d admin admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:54.099734 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b9f2d1-a004-4fd1-884a-ff0fd3c8fe8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:54.101952 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b9f2d1-a004-4fd1-884a-ff0fd3c8fe8d admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:54.218366 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ca23d-6d67-49f8-9182-6246464498b1 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:54.233245 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 888/1788] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:56:54 2022] POST /identity/v3/projects => generated 361 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:56:54.235366 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ca23d-6d67-49f8-9182-6246464498b1 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:54.413278 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b9f2d1-a004-4fd1-884a-ff0fd3c8fe8d admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:54.415168 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-83b9f2d1-a004-4fd1-884a-ff0fd3c8fe8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:54.423168 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:54.699747 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b9f2d1-a004-4fd1-884a-ff0fd3c8fe8d admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:54.713842 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12afec7-573c-4827-8da6-3c761bc303ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:54.715044 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12afec7-573c-4827-8da6-3c761bc303ef service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:54.715672 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12afec7-573c-4827-8da6-3c761bc303ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:54.716091 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12afec7-573c-4827-8da6-3c761bc303ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:54.727336 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12afec7-573c-4827-8da6-3c761bc303ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:54.732066 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12afec7-573c-4827-8da6-3c761bc303ef service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:54.792074 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 889/1789] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:56:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:56:55.119253 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad79312-f032-49fc-97b3-1c500dc192d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:55.120590 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad79312-f032-49fc-97b3-1c500dc192d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:55.127911 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad79312-f032-49fc-97b3-1c500dc192d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:55.128429 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad79312-f032-49fc-97b3-1c500dc192d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:55.129434 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad79312-f032-49fc-97b3-1c500dc192d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:55.131722 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad79312-f032-49fc-97b3-1c500dc192d4 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:55.175452 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 901/1790] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:56:54 2022] POST /identity/v3/users => generated 281 bytes in 958 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:56:55.177649 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12afec7-573c-4827-8da6-3c761bc303ef service neutron] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:55.208078 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12afec7-573c-4827-8da6-3c761bc303ef service neutron] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:55.234035 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc8f252-da0a-4310-891f-fcc07abc8cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:55.235186 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8f252-da0a-4310-891f-fcc07abc8cdf admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:55.235541 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8f252-da0a-4310-891f-fcc07abc8cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:55.235902 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8f252-da0a-4310-891f-fcc07abc8cdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:55.237807 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc8f252-da0a-4310-891f-fcc07abc8cdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:55.249971 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc8f252-da0a-4310-891f-fcc07abc8cdf admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:55.281344 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 890/1791] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:56:55 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 01:56:55.282686 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad79312-f032-49fc-97b3-1c500dc192d4 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:55.303179 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad79312-f032-49fc-97b3-1c500dc192d4 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:55.309106 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7671fb97-be69-48a3-b044-e3a77647edb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:55.310115 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671fb97-be69-48a3-b044-e3a77647edb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:55.310553 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671fb97-be69-48a3-b044-e3a77647edb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:55.310989 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671fb97-be69-48a3-b044-e3a77647edb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:55.311913 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7671fb97-be69-48a3-b044-e3a77647edb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:55.313865 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7671fb97-be69-48a3-b044-e3a77647edb3 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:55.340429 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 902/1792] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:56:55 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:56:55.344770 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc8f252-da0a-4310-891f-fcc07abc8cdf admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:55.365992 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc8f252-da0a-4310-891f-fcc07abc8cdf admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:55.372227 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5900bb14-dc17-4769-b1fa-b299f9a0974f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:55.373535 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5900bb14-dc17-4769-b1fa-b299f9a0974f admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:55.373983 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5900bb14-dc17-4769-b1fa-b299f9a0974f admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:55.374416 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5900bb14-dc17-4769-b1fa-b299f9a0974f admin admin] PATH_INFO: `/v3/projects/b2b6765d13ed46b985141a686ef8409d/users/4a55fe5d351f4e7781c4ba8f8cf70761/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:55.401384 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb63e75-ec14-47b7-9d87-9f1bdec0fae0 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:55.402071 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb63e75-ec14-47b7-9d87-9f1bdec0fae0 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:55.402576 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb63e75-ec14-47b7-9d87-9f1bdec0fae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:55.435972 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5900bb14-dc17-4769-b1fa-b299f9a0974f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2b6765d13ed46b985141a686ef8409d, user_id=4a55fe5d351f4e7781c4ba8f8cf70761, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:55.439730 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5900bb14-dc17-4769-b1fa-b299f9a0974f admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:55.441133 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-5bb63e75-ec14-47b7-9d87-9f1bdec0fae0 None None] MFA Rules not processed for user `ababea3e976f46889f2b1980edfee883`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:55.445380 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-5bb63e75-ec14-47b7-9d87-9f1bdec0fae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:55.476233 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 891/1793] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:56:55 2022] PUT /identity/v3/projects/b2b6765d13ed46b985141a686ef8409d/users/4a55fe5d351f4e7781c4ba8f8cf70761/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:56:55.487133 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:55.489528 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3746cf3e-0660-4d5e-9356-3f40a361bf56 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:55.490092 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3746cf3e-0660-4d5e-9356-3f40a361bf56 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:55.492883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3746cf3e-0660-4d5e-9356-3f40a361bf56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:55.574276 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-3746cf3e-0660-4d5e-9356-3f40a361bf56 None None] MFA Rules not processed for user `4a55fe5d351f4e7781c4ba8f8cf70761`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:55.582558 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-3746cf3e-0660-4d5e-9356-3f40a361bf56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:55.653713 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 903/1794] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:55.800136 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 892/1795] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:55 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:55.805011 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb63e75-ec14-47b7-9d87-9f1bdec0fae0 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:55.820742 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb63e75-ec14-47b7-9d87-9f1bdec0fae0 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:56.003359 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff351e-7328-46e1-9039-34d73809d92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:56.004090 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff351e-7328-46e1-9039-34d73809d92c admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:56.006859 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff351e-7328-46e1-9039-34d73809d92c admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:56.007422 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff351e-7328-46e1-9039-34d73809d92c admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:56.008363 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff351e-7328-46e1-9039-34d73809d92c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:56.011996 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff351e-7328-46e1-9039-34d73809d92c admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:56.123846 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 904/1796] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:56:55 2022] POST /identity/v3/projects => generated 363 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:56:56.125054 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3746cf3e-0660-4d5e-9356-3f40a361bf56 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:56.144026 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3746cf3e-0660-4d5e-9356-3f40a361bf56 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:56.411515 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f117ea6-dc44-4de0-98d9-73fd6be3112f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:56.413009 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f117ea6-dc44-4de0-98d9-73fd6be3112f admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:56.413489 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f117ea6-dc44-4de0-98d9-73fd6be3112f admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:56.413923 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f117ea6-dc44-4de0-98d9-73fd6be3112f admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:56.414842 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f117ea6-dc44-4de0-98d9-73fd6be3112f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:56.420852 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f117ea6-dc44-4de0-98d9-73fd6be3112f admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:56.443507 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:56.480341 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 893/1797] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:56:56 2022] POST /identity/v3/users => generated 288 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:56:56.484709 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff351e-7328-46e1-9039-34d73809d92c admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:56.502625 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff351e-7328-46e1-9039-34d73809d92c admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:56.521644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636fc4ee-e68e-4258-8446-bca8ab250acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:56.523423 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636fc4ee-e68e-4258-8446-bca8ab250acf admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:56.523793 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636fc4ee-e68e-4258-8446-bca8ab250acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:56.529357 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636fc4ee-e68e-4258-8446-bca8ab250acf admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:56.530246 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636fc4ee-e68e-4258-8446-bca8ab250acf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:56.542922 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636fc4ee-e68e-4258-8446-bca8ab250acf admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:56.597712 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 905/1798] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:56:56 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:56:56.602392 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f117ea6-dc44-4de0-98d9-73fd6be3112f admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:56.622675 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f117ea6-dc44-4de0-98d9-73fd6be3112f admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:56.641494 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd89cd2-d593-49fa-bfa7-ebc30c5d5500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:56.642565 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd89cd2-d593-49fa-bfa7-ebc30c5d5500 admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:56.642853 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd89cd2-d593-49fa-bfa7-ebc30c5d5500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:56.643132 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd89cd2-d593-49fa-bfa7-ebc30c5d5500 admin admin] PATH_INFO: `/v3/projects/6305c5cedc0a4f51accfaae2ffe8e3d1/users/4949d25a48a04780857999b19b3f8b65/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:56.699980 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd89cd2-d593-49fa-bfa7-ebc30c5d5500 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6305c5cedc0a4f51accfaae2ffe8e3d1, user_id=4949d25a48a04780857999b19b3f8b65, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:56.709251 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd89cd2-d593-49fa-bfa7-ebc30c5d5500 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:56.740766 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 894/1799] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:56:56 2022] PUT /identity/v3/projects/6305c5cedc0a4f51accfaae2ffe8e3d1/users/4949d25a48a04780857999b19b3f8b65/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:56:56.744815 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636fc4ee-e68e-4258-8446-bca8ab250acf admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:56.763713 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636fc4ee-e68e-4258-8446-bca8ab250acf admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:56.999757 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6167b0-5679-47e9-b788-1956b3845c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:57.004534 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6167b0-5679-47e9-b788-1956b3845c0b admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:57.005944 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6167b0-5679-47e9-b788-1956b3845c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:57.007834 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6167b0-5679-47e9-b788-1956b3845c0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:57.009536 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6167b0-5679-47e9-b788-1956b3845c0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:57.016421 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6167b0-5679-47e9-b788-1956b3845c0b admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:57.060602 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd89cd2-d593-49fa-bfa7-ebc30c5d5500 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:57.063137 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-1dd89cd2-d593-49fa-bfa7-ebc30c5d5500 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:57.075159 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 906/1800] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:56:56 2022] GET /identity/v3/roles => generated 1420 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:56:57.078025 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6167b0-5679-47e9-b788-1956b3845c0b admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:57.089918 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd89cd2-d593-49fa-bfa7-ebc30c5d5500 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:57.129215 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6167b0-5679-47e9-b788-1956b3845c0b admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:57.131258 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe05054-a4fb-4dd4-8ece-f67b55c07b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:57.139747 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe05054-a4fb-4dd4-8ece-f67b55c07b61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:57.140280 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe05054-a4fb-4dd4-8ece-f67b55c07b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:57.141588 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe05054-a4fb-4dd4-8ece-f67b55c07b61 admin admin] PATH_INFO: `/v3/projects/ce8d9cda737a4812a53fa2a528214f1b` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:57.145227 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe05054-a4fb-4dd4-8ece-f67b55c07b61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce8d9cda737a4812a53fa2a528214f1b)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:57.152807 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe05054-a4fb-4dd4-8ece-f67b55c07b61 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:57.162127 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52acf1a7-d394-49e6-a7cc-e8db6aef693a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:57.163107 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52acf1a7-d394-49e6-a7cc-e8db6aef693a admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:57.163354 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52acf1a7-d394-49e6-a7cc-e8db6aef693a admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:57.163629 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52acf1a7-d394-49e6-a7cc-e8db6aef693a admin admin] PATH_INFO: `/v3/projects/6305c5cedc0a4f51accfaae2ffe8e3d1/users/4949d25a48a04780857999b19b3f8b65/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:57.170143 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52acf1a7-d394-49e6-a7cc-e8db6aef693a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6305c5cedc0a4f51accfaae2ffe8e3d1, user_id=4949d25a48a04780857999b19b3f8b65, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:57.185729 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52acf1a7-d394-49e6-a7cc-e8db6aef693a admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:57.224927 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 907/1801] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:56:57 2022] PUT /identity/v3/projects/6305c5cedc0a4f51accfaae2ffe8e3d1/users/4949d25a48a04780857999b19b3f8b65/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:56:57.235368 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b9608a-4ebe-4a16-8e9f-9cbe8928126c None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:57.235659 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b9608a-4ebe-4a16-8e9f-9cbe8928126c None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:57.235936 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b9608a-4ebe-4a16-8e9f-9cbe8928126c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:57.314521 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-abe05054-a4fb-4dd4-8ece-f67b55c07b61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce8d9cda737a4812a53fa2a528214f1b', 'request_id': 'req-abe05054-a4fb-4dd4-8ece-f67b55c07b61'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:57.332795 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-abe05054-a4fb-4dd4-8ece-f67b55c07b61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce8d9cda737a4812a53fa2a528214f1b', 'request_id': 'req-abe05054-a4fb-4dd4-8ece-f67b55c07b61'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:57.343130 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-07b9608a-4ebe-4a16-8e9f-9cbe8928126c None None] MFA Rules not processed for user `4949d25a48a04780857999b19b3f8b65`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:57.349317 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-07b9608a-4ebe-4a16-8e9f-9cbe8928126c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:57.357091 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-abe05054-a4fb-4dd4-8ece-f67b55c07b61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce8d9cda737a4812a53fa2a528214f1b', 'request_id': 'req-abe05054-a4fb-4dd4-8ece-f67b55c07b61'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:57.388251 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:57.398596 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-abe05054-a4fb-4dd4-8ece-f67b55c07b61 admin admin] The token cache is being invalidate because project ce8d9cda737a4812a53fa2a528214f1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:56:57.398926 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-abe05054-a4fb-4dd4-8ece-f67b55c07b61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:57.399985 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-abe05054-a4fb-4dd4-8ece-f67b55c07b61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:56:57.404918 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 895/1802] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:56:57 2022] DELETE /identity/v3/projects/ce8d9cda737a4812a53fa2a528214f1b => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:56:57.409415 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5650fc-93e8-41f7-8717-f0ecfff4c001 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:57.410506 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5650fc-93e8-41f7-8717-f0ecfff4c001 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:57.410944 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5650fc-93e8-41f7-8717-f0ecfff4c001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:57.448467 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-ef5650fc-93e8-41f7-8717-f0ecfff4c001 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:57.451267 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-ef5650fc-93e8-41f7-8717-f0ecfff4c001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:57.637086 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 908/1803] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:57 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 403 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:57.652648 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b9608a-4ebe-4a16-8e9f-9cbe8928126c None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:57.655248 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-07b9608a-4ebe-4a16-8e9f-9cbe8928126c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:57.659460 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 896/1804] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:57.668537 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233da12e-5651-48fb-b803-4ed33864936b None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:57.671302 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233da12e-5651-48fb-b803-4ed33864936b None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:57.671593 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233da12e-5651-48fb-b803-4ed33864936b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:57.738992 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-233da12e-5651-48fb-b803-4ed33864936b None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:57.740449 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-233da12e-5651-48fb-b803-4ed33864936b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:57.772031 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 897/1805] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 01:56:57.774301 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233da12e-5651-48fb-b803-4ed33864936b None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:57.808852 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233da12e-5651-48fb-b803-4ed33864936b None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:57.830752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c3ba0-ee49-407d-9639-b4a15a883c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:57.832443 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c3ba0-ee49-407d-9639-b4a15a883c6a admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:57.833466 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c3ba0-ee49-407d-9639-b4a15a883c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:57.834855 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c3ba0-ee49-407d-9639-b4a15a883c6a admin admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:57.835915 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c3ba0-ee49-407d-9639-b4a15a883c6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:56:57.843303 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c3ba0-ee49-407d-9639-b4a15a883c6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:57.845405 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c3ba0-ee49-407d-9639-b4a15a883c6a admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:57.867433 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b9608a-4ebe-4a16-8e9f-9cbe8928126c None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:57.882779 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764e0d06-e050-47b1-a9ad-dbbc9c3a29d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:57.883508 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e0d06-e050-47b1-a9ad-dbbc9c3a29d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:57.883791 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e0d06-e050-47b1-a9ad-dbbc9c3a29d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:57.884062 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e0d06-e050-47b1-a9ad-dbbc9c3a29d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:57.893449 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 898/1806] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 01:56:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:56:57.899051 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c3ba0-ee49-407d-9639-b4a15a883c6a admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:57.932010 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764e0d06-e050-47b1-a9ad-dbbc9c3a29d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:57.933611 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764e0d06-e050-47b1-a9ad-dbbc9c3a29d8 service cinder] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:57.938645 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c3ba0-ee49-407d-9639-b4a15a883c6a admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:57.963080 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a259f-d907-4104-b192-2f8bbd4047de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:57.964031 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a259f-d907-4104-b192-2f8bbd4047de admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:57.966937 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a259f-d907-4104-b192-2f8bbd4047de admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:57.968416 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a259f-d907-4104-b192-2f8bbd4047de admin admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:57.970341 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9a259f-d907-4104-b192-2f8bbd4047de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:57.976649 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9a259f-d907-4104-b192-2f8bbd4047de admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:57.979655 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 909/1807] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:56:57 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:56:58.056064 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 899/1808] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:56:57 2022] POST /identity/v3/projects => generated 355 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:56:58.059703 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764e0d06-e050-47b1-a9ad-dbbc9c3a29d8 service cinder] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:58.078747 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764e0d06-e050-47b1-a9ad-dbbc9c3a29d8 service cinder] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:58.152877 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a259f-d907-4104-b192-2f8bbd4047de admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:58.154787 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-5e9a259f-d907-4104-b192-2f8bbd4047de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:58.195467 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:58.269861 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07f75d1-10ea-48c4-8457-03e3083e0014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:58.270356 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f75d1-10ea-48c4-8457-03e3083e0014 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:58.270620 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f75d1-10ea-48c4-8457-03e3083e0014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:58.270899 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f75d1-10ea-48c4-8457-03e3083e0014 admin admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:58.271705 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07f75d1-10ea-48c4-8457-03e3083e0014 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:58.273650 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07f75d1-10ea-48c4-8457-03e3083e0014 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:58.330078 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a259f-d907-4104-b192-2f8bbd4047de admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:58.337169 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 910/1809] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:56:58 2022] POST /identity/v3/users => generated 278 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:56:58.339565 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07f75d1-10ea-48c4-8457-03e3083e0014 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:58.345327 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0069bc-ccf7-4752-972c-480a6381c7be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:58.346737 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0069bc-ccf7-4752-972c-480a6381c7be service cinder] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:58.347021 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0069bc-ccf7-4752-972c-480a6381c7be service cinder] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:58.347323 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0069bc-ccf7-4752-972c-480a6381c7be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:58.353372 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-8e0069bc-ccf7-4752-972c-480a6381c7be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:58.367751 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07f75d1-10ea-48c4-8457-03e3083e0014 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:58.375492 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090bbefd-e970-4bab-85c1-635acc70e144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:58.376226 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090bbefd-e970-4bab-85c1-635acc70e144 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:58.376556 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090bbefd-e970-4bab-85c1-635acc70e144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:58.376837 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090bbefd-e970-4bab-85c1-635acc70e144 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:58.377580 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090bbefd-e970-4bab-85c1-635acc70e144 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:58.379636 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090bbefd-e970-4bab-85c1-635acc70e144 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:58.398633 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 911/1810] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:56:58 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:56:58.406078 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090bbefd-e970-4bab-85c1-635acc70e144 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:58.451831 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090bbefd-e970-4bab-85c1-635acc70e144 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:58.464397 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a158d1-7846-42c2-88a4-44f202919410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:58.465651 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a158d1-7846-42c2-88a4-44f202919410 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:58.466112 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a158d1-7846-42c2-88a4-44f202919410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:58.466574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a158d1-7846-42c2-88a4-44f202919410 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:58.467520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a158d1-7846-42c2-88a4-44f202919410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:58.469407 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a158d1-7846-42c2-88a4-44f202919410 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:58.483649 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 912/1811] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:56:58 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:56:58.487518 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a158d1-7846-42c2-88a4-44f202919410 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:58.515854 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a158d1-7846-42c2-88a4-44f202919410 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:58.523153 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e726805b-10ce-49a9-921c-74481befc852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:58.524396 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e726805b-10ce-49a9-921c-74481befc852 admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:58.524952 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e726805b-10ce-49a9-921c-74481befc852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:58.525495 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e726805b-10ce-49a9-921c-74481befc852 admin admin] PATH_INFO: `/v3/projects/15facda1b2534a72852d00b61b63bc1c/users/6828d54b02ad497d80e57f9b0f4fbe95/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:58.558324 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e726805b-10ce-49a9-921c-74481befc852 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15facda1b2534a72852d00b61b63bc1c, user_id=6828d54b02ad497d80e57f9b0f4fbe95, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:58.569094 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e726805b-10ce-49a9-921c-74481befc852 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:58.577466 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:56:58.600958 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 913/1812] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:56:58 2022] PUT /identity/v3/projects/15facda1b2534a72852d00b61b63bc1c/users/6828d54b02ad497d80e57f9b0f4fbe95/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:56:58.617909 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fdd575-91da-4206-9f3d-81be62914597 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:58.620506 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fdd575-91da-4206-9f3d-81be62914597 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:58.620892 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fdd575-91da-4206-9f3d-81be62914597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:58.628038 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0069bc-ccf7-4752-972c-480a6381c7be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:58.638103 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0069bc-ccf7-4752-972c-480a6381c7be service cinder] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:58.688791 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 900/1813] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:56:58 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 01:56:58.739294 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-44fdd575-91da-4206-9f3d-81be62914597 None None] MFA Rules not processed for user `6828d54b02ad497d80e57f9b0f4fbe95`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:56:58.740824 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-44fdd575-91da-4206-9f3d-81be62914597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:58.976932 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0069bc-ccf7-4752-972c-480a6381c7be service cinder] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:58.980569 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 914/1814] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:56:58 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:56:58.991736 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0069bc-ccf7-4752-972c-480a6381c7be service cinder] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:59.215710 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2129f937-ae96-416c-80c0-e3c9e7419938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:59.216412 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129f937-ae96-416c-80c0-e3c9e7419938 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:59.216829 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129f937-ae96-416c-80c0-e3c9e7419938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:59.216995 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129f937-ae96-416c-80c0-e3c9e7419938 admin admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:59.217677 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2129f937-ae96-416c-80c0-e3c9e7419938 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:59.219707 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2129f937-ae96-416c-80c0-e3c9e7419938 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:59.341667 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 901/1815] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:56:58 2022] POST /identity/v3/projects => generated 357 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 01:56:59.347336 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fdd575-91da-4206-9f3d-81be62914597 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:59.378818 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fdd575-91da-4206-9f3d-81be62914597 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:59.619100 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b08bf-c767-4737-bae3-5578876fa51b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:59.620357 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b08bf-c767-4737-bae3-5578876fa51b admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:59.622225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b08bf-c767-4737-bae3-5578876fa51b admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:59.622618 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b08bf-c767-4737-bae3-5578876fa51b admin admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:59.623522 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958b08bf-c767-4737-bae3-5578876fa51b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:59.629779 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958b08bf-c767-4737-bae3-5578876fa51b admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:59.687004 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 915/1816] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:56:59 2022] POST /identity/v3/users => generated 285 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:56:59.691660 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2129f937-ae96-416c-80c0-e3c9e7419938 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:59.718199 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2129f937-ae96-416c-80c0-e3c9e7419938 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:59.744307 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b08bf-c767-4737-bae3-5578876fa51b admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:59.746497 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-958b08bf-c767-4737-bae3-5578876fa51b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:56:59.770295 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90c0726-342e-4136-abd2-b66df419644e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:59.774185 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c0726-342e-4136-abd2-b66df419644e admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:59.775961 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c0726-342e-4136-abd2-b66df419644e admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:59.776849 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c0726-342e-4136-abd2-b66df419644e admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:59.777674 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b08bf-c767-4737-bae3-5578876fa51b admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:59.779206 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90c0726-342e-4136-abd2-b66df419644e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:59.784558 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90c0726-342e-4136-abd2-b66df419644e admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:59.820053 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe47a56b-1b5e-4fbf-aeef-92a699d689b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:59.821089 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47a56b-1b5e-4fbf-aeef-92a699d689b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:59.821368 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47a56b-1b5e-4fbf-aeef-92a699d689b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:59.821644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47a56b-1b5e-4fbf-aeef-92a699d689b7 admin admin] PATH_INFO: `/v3/users/cdeaf0128bee46b1a595d96bba6a2817` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:59.824366 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe47a56b-1b5e-4fbf-aeef-92a699d689b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdeaf0128bee46b1a595d96bba6a2817)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:59.826355 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe47a56b-1b5e-4fbf-aeef-92a699d689b7 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:59.830200 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 902/1817] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:56:59 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:56:59.831292 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90c0726-342e-4136-abd2-b66df419644e admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:56:59.869844 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90c0726-342e-4136-abd2-b66df419644e admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:56:59.881049 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5de281-e8d9-4059-945b-4f3ad7bb1037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:56:59.883401 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5de281-e8d9-4059-945b-4f3ad7bb1037 admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:56:59.883591 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5de281-e8d9-4059-945b-4f3ad7bb1037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:56:59.888268 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5de281-e8d9-4059-945b-4f3ad7bb1037 admin admin] PATH_INFO: `/v3/projects/2b3fbb7abbad469c81be764a49aa5889/users/d824028753bc4e2eb7284c93274eb65a/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:56:59.948319 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5de281-e8d9-4059-945b-4f3ad7bb1037 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b3fbb7abbad469c81be764a49aa5889, user_id=d824028753bc4e2eb7284c93274eb65a, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:56:59.956095 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5de281-e8d9-4059-945b-4f3ad7bb1037 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:56:59.973807 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-fe47a56b-1b5e-4fbf-aeef-92a699d689b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdeaf0128bee46b1a595d96bba6a2817'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:00.022830 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 903/1818] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:56:59 2022] PUT /identity/v3/projects/2b3fbb7abbad469c81be764a49aa5889/users/d824028753bc4e2eb7284c93274eb65a/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:00.031074 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5de281-e8d9-4059-945b-4f3ad7bb1037 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:00.066832 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-fe47a56b-1b5e-4fbf-aeef-92a699d689b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdeaf0128bee46b1a595d96bba6a2817'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:00.072707 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:INSERT=5,keystone:SELECT=86,keystone:DELETE=9 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:00.092806 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-fe47a56b-1b5e-4fbf-aeef-92a699d689b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdeaf0128bee46b1a595d96bba6a2817'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:00.103290 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:00.131677 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-fe47a56b-1b5e-4fbf-aeef-92a699d689b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdeaf0128bee46b1a595d96bba6a2817'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:00.134313 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-fe47a56b-1b5e-4fbf-aeef-92a699d689b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdeaf0128bee46b1a595d96bba6a2817'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:00.159715 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 916/1819] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:56:59 2022] DELETE /identity/v3/users/cdeaf0128bee46b1a595d96bba6a2817 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:00.161119 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db98fc-271e-422c-9488-56ab5214675b None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:00.161409 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db98fc-271e-422c-9488-56ab5214675b None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:00.161744 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db98fc-271e-422c-9488-56ab5214675b None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:00.163080 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 917/1820] 10.222.0.26 () {54 vars in 921 bytes} [Thu Jun 23 01:57:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:57:00.186285 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17361c5-2ab6-4bff-aa47-42a9e0025d8c None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:00.186620 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17361c5-2ab6-4bff-aa47-42a9e0025d8c None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:00.186971 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17361c5-2ab6-4bff-aa47-42a9e0025d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:00.291279 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-f17361c5-2ab6-4bff-aa47-42a9e0025d8c None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:00.291279 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-f17361c5-2ab6-4bff-aa47-42a9e0025d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:00.387812 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5de281-e8d9-4059-945b-4f3ad7bb1037 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:00.650015 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 918/1821] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 486 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:00.651504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d87cec-ab72-4f3f-a1b7-1bdaf64f4131 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:00.651766 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d87cec-ab72-4f3f-a1b7-1bdaf64f4131 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:00.652013 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d87cec-ab72-4f3f-a1b7-1bdaf64f4131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:00.734107 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-90d87cec-ab72-4f3f-a1b7-1bdaf64f4131 None None] MFA Rules not processed for user `ae4e83df33b64a3ebf28f54e2d94fe63`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:00.737970 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-90d87cec-ab72-4f3f-a1b7-1bdaf64f4131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:00.746599 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b231548a-1a41-4cbc-a5f3-30bca8731bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:00.748693 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b231548a-1a41-4cbc-a5f3-30bca8731bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:00.749713 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b231548a-1a41-4cbc-a5f3-30bca8731bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:00.751908 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b231548a-1a41-4cbc-a5f3-30bca8731bc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:00.754577 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b231548a-1a41-4cbc-a5f3-30bca8731bc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:00.764401 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b231548a-1a41-4cbc-a5f3-30bca8731bc8 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:00.830302 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 904/1822] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 797 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:57:00.832295 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b231548a-1a41-4cbc-a5f3-30bca8731bc8 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:00.839450 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-b231548a-1a41-4cbc-a5f3-30bca8731bc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:01.130431 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 919/1823] 10.222.0.26 () {58 vars in 1018 bytes} [Thu Jun 23 01:57:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 478 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:01.131186 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d87cec-ab72-4f3f-a1b7-1bdaf64f4131 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:01.137250 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-90d87cec-ab72-4f3f-a1b7-1bdaf64f4131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:01.160824 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b231548a-1a41-4cbc-a5f3-30bca8731bc8 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:01.174158 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40bfbf8-c9c2-49c9-b259-4b4b0c2adee0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:01.175288 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40bfbf8-c9c2-49c9-b259-4b4b0c2adee0 service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:01.175617 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40bfbf8-c9c2-49c9-b259-4b4b0c2adee0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:01.175892 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40bfbf8-c9c2-49c9-b259-4b4b0c2adee0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:01.184015 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d87cec-ab72-4f3f-a1b7-1bdaf64f4131 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:01.207478 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc8501d-1a4c-4078-b9a4-667f29cca607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:01.216849 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc8501d-1a4c-4078-b9a4-667f29cca607 admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:01.219166 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc8501d-1a4c-4078-b9a4-667f29cca607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:01.221571 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc8501d-1a4c-4078-b9a4-667f29cca607 admin admin] PATH_INFO: `/v3/projects/2b3fbb7abbad469c81be764a49aa5889/users/d824028753bc4e2eb7284c93274eb65a/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:01.225419 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40bfbf8-c9c2-49c9-b259-4b4b0c2adee0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:01.234388 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40bfbf8-c9c2-49c9-b259-4b4b0c2adee0 service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:01.237689 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc8501d-1a4c-4078-b9a4-667f29cca607 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b3fbb7abbad469c81be764a49aa5889, user_id=d824028753bc4e2eb7284c93274eb65a, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:01.249820 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc8501d-1a4c-4078-b9a4-667f29cca607 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:01.283621 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 905/1824] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:57:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:57:01.296451 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 920/1825] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:57:01 2022] PUT /identity/v3/projects/2b3fbb7abbad469c81be764a49aa5889/users/d824028753bc4e2eb7284c93274eb65a/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:01.310591 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e9ef3-f5ff-475d-ac09-86aaa32f6b37 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:01.310869 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e9ef3-f5ff-475d-ac09-86aaa32f6b37 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:01.311406 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e9ef3-f5ff-475d-ac09-86aaa32f6b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:01.412505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc8501d-1a4c-4078-b9a4-667f29cca607 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:01.414237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-ffc8501d-1a4c-4078-b9a4-667f29cca607 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:01.420508 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-ad1e9ef3-f5ff-475d-ac09-86aaa32f6b37 None None] MFA Rules not processed for user `d824028753bc4e2eb7284c93274eb65a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:01.421854 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-ad1e9ef3-f5ff-475d-ac09-86aaa32f6b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:01.661966 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:01.730339 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:01.753397 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc8501d-1a4c-4078-b9a4-667f29cca607 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:01.778484 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4f692-6428-4628-86c0-7b1b143ffb06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4e83df33b64a3ebf28f54e2d94fe63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:01.783106 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4f692-6428-4628-86c0-7b1b143ffb06 service nova] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:01.785339 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4f692-6428-4628-86c0-7b1b143ffb06 service nova] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:01.786217 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4f692-6428-4628-86c0-7b1b143ffb06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:01.799866 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 906/1826] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:01 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 486 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:01.808032 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1e9ef3-f5ff-475d-ac09-86aaa32f6b37 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:01.809589 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-ad1e9ef3-f5ff-475d-ac09-86aaa32f6b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:01.820896 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e4f692-6428-4628-86c0-7b1b143ffb06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:01.823707 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e4f692-6428-4628-86c0-7b1b143ffb06 service nova] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:01.874397 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 921/1827] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:57:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 01:57:01.924012 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1e9ef3-f5ff-475d-ac09-86aaa32f6b37 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:01.932267 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b17dd4-d684-45fd-b8c4-8f6b8ad2734c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:01.933197 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b17dd4-d684-45fd-b8c4-8f6b8ad2734c service cinder] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:01.934788 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b17dd4-d684-45fd-b8c4-8f6b8ad2734c service cinder] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:01.936011 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b17dd4-d684-45fd-b8c4-8f6b8ad2734c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:01.956703 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b17dd4-d684-45fd-b8c4-8f6b8ad2734c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:01.958224 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b17dd4-d684-45fd-b8c4-8f6b8ad2734c service cinder] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:01.988876 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 907/1828] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:57:01 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:57:02.159748 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4f692-6428-4628-86c0-7b1b143ffb06 service nova] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:02.172743 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4f692-6428-4628-86c0-7b1b143ffb06 service nova] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:02.187729 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d541579d-c875-458f-a617-e8027583d95e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:02.190390 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541579d-c875-458f-a617-e8027583d95e service cinder] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:02.191256 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541579d-c875-458f-a617-e8027583d95e service cinder] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:02.191459 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541579d-c875-458f-a617-e8027583d95e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:02.203145 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-d541579d-c875-458f-a617-e8027583d95e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:02.379677 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d541579d-c875-458f-a617-e8027583d95e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:02.381797 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d541579d-c875-458f-a617-e8027583d95e service cinder] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:02.410504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 922/1829] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:57:02 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:57:02.448050 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a155d15-c190-47cf-9cf9-0d1ef5c10797 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:02.448331 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a155d15-c190-47cf-9cf9-0d1ef5c10797 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:02.448819 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a155d15-c190-47cf-9cf9-0d1ef5c10797 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:02.450666 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 908/1830] 10.222.0.26 () {54 vars in 921 bytes} [Thu Jun 23 01:57:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:57:02.486617 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def99e42-410e-4965-9fb2-9f30d171b97d None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:02.487325 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def99e42-410e-4965-9fb2-9f30d171b97d None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:02.487853 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def99e42-410e-4965-9fb2-9f30d171b97d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:02.551580 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-def99e42-410e-4965-9fb2-9f30d171b97d None None] MFA Rules not processed for user `ae4e83df33b64a3ebf28f54e2d94fe63`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:02.554149 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-def99e42-410e-4965-9fb2-9f30d171b97d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:02.566418 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 923/1831] 10.222.0.26 () {58 vars in 1018 bytes} [Thu Jun 23 01:57:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:02.592316 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a155d15-c190-47cf-9cf9-0d1ef5c10797 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:02.594402 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-8a155d15-c190-47cf-9cf9-0d1ef5c10797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:02.642950 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a155d15-c190-47cf-9cf9-0d1ef5c10797 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:02.663819 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de89709-5b9b-4659-9636-26149cb8ed8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4e83df33b64a3ebf28f54e2d94fe63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:02.664837 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de89709-5b9b-4659-9636-26149cb8ed8b service nova] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:02.664837 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de89709-5b9b-4659-9636-26149cb8ed8b service nova] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:02.665026 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de89709-5b9b-4659-9636-26149cb8ed8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:02.680281 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de89709-5b9b-4659-9636-26149cb8ed8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:02.682612 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de89709-5b9b-4659-9636-26149cb8ed8b service nova] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:02.696961 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 909/1832] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:57:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:57:02.906985 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def99e42-410e-4965-9fb2-9f30d171b97d None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:02.912162 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-def99e42-410e-4965-9fb2-9f30d171b97d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:02.995125 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def99e42-410e-4965-9fb2-9f30d171b97d None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:03.001095 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a315bf48-ae38-4aa0-b719-69e424d0f9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:03.002501 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a315bf48-ae38-4aa0-b719-69e424d0f9a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:03.002936 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a315bf48-ae38-4aa0-b719-69e424d0f9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:03.003419 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a315bf48-ae38-4aa0-b719-69e424d0f9a5 admin admin] PATH_INFO: `/v3/users/4a55fe5d351f4e7781c4ba8f8cf70761` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:03.004823 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a315bf48-ae38-4aa0-b719-69e424d0f9a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a55fe5d351f4e7781c4ba8f8cf70761)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:03.008718 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a315bf48-ae38-4aa0-b719-69e424d0f9a5 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:03.066319 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-a315bf48-ae38-4aa0-b719-69e424d0f9a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a55fe5d351f4e7781c4ba8f8cf70761'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:03.087535 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-a315bf48-ae38-4aa0-b719-69e424d0f9a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a55fe5d351f4e7781c4ba8f8cf70761'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:03.101921 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-a315bf48-ae38-4aa0-b719-69e424d0f9a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a55fe5d351f4e7781c4ba8f8cf70761'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:03.117912 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-a315bf48-ae38-4aa0-b719-69e424d0f9a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a55fe5d351f4e7781c4ba8f8cf70761'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:03.118863 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-a315bf48-ae38-4aa0-b719-69e424d0f9a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a55fe5d351f4e7781c4ba8f8cf70761'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:03.125058 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:03.137950 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 924/1833] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:57:02 2022] DELETE /identity/v3/users/4a55fe5d351f4e7781c4ba8f8cf70761 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:03.142036 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f3a4b-6602-4b61-ac27-ed9e3d5b4bb7 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:03.142622 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f3a4b-6602-4b61-ac27-ed9e3d5b4bb7 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:03.142622 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f3a4b-6602-4b61-ac27-ed9e3d5b4bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:03.162214 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-604f3a4b-6602-4b61-ac27-ed9e3d5b4bb7 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:03.163822 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-604f3a4b-6602-4b61-ac27-ed9e3d5b4bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:03.280153 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 910/1834] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:03.288847 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a315bf48-ae38-4aa0-b719-69e424d0f9a5 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:03.296510 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-a315bf48-ae38-4aa0-b719-69e424d0f9a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:03.304948 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604f3a4b-6602-4b61-ac27-ed9e3d5b4bb7 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:03.307449 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-604f3a4b-6602-4b61-ac27-ed9e3d5b4bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:03.354348 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604f3a4b-6602-4b61-ac27-ed9e3d5b4bb7 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:03.362469 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82928bc0-1398-45b6-80fc-4c2a36998e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:03.364115 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82928bc0-1398-45b6-80fc-4c2a36998e2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:03.365440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82928bc0-1398-45b6-80fc-4c2a36998e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:03.365440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82928bc0-1398-45b6-80fc-4c2a36998e2a admin admin] PATH_INFO: `/v3/projects/b63e012e8eaa4c91aa1d2cbac60f7490` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:03.366622 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82928bc0-1398-45b6-80fc-4c2a36998e2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b63e012e8eaa4c91aa1d2cbac60f7490)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:03.367134 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a315bf48-ae38-4aa0-b719-69e424d0f9a5 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:03.368628 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82928bc0-1398-45b6-80fc-4c2a36998e2a admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:03.375888 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23276de9-d604-4c4e-8754-d1bdb0ab50a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:03.376688 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23276de9-d604-4c4e-8754-d1bdb0ab50a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:03.377073 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23276de9-d604-4c4e-8754-d1bdb0ab50a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:03.377359 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23276de9-d604-4c4e-8754-d1bdb0ab50a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:03.393481 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23276de9-d604-4c4e-8754-d1bdb0ab50a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:03.396405 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23276de9-d604-4c4e-8754-d1bdb0ab50a3 service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:03.409454 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 925/1835] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:57:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 01:57:03.445090 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-82928bc0-1398-45b6-80fc-4c2a36998e2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b63e012e8eaa4c91aa1d2cbac60f7490', 'request_id': 'req-82928bc0-1398-45b6-80fc-4c2a36998e2a'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:03.452829 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-82928bc0-1398-45b6-80fc-4c2a36998e2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b63e012e8eaa4c91aa1d2cbac60f7490', 'request_id': 'req-82928bc0-1398-45b6-80fc-4c2a36998e2a'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:03.466218 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-82928bc0-1398-45b6-80fc-4c2a36998e2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b63e012e8eaa4c91aa1d2cbac60f7490', 'request_id': 'req-82928bc0-1398-45b6-80fc-4c2a36998e2a'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:03.478717 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:03.486721 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-82928bc0-1398-45b6-80fc-4c2a36998e2a admin admin] The token cache is being invalidate because project b63e012e8eaa4c91aa1d2cbac60f7490 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:57:03.487374 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-82928bc0-1398-45b6-80fc-4c2a36998e2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:03.489261 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-82928bc0-1398-45b6-80fc-4c2a36998e2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:03.492472 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 911/1836] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:57:03 2022] DELETE /identity/v3/projects/b63e012e8eaa4c91aa1d2cbac60f7490 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 01:57:03.497377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23276de9-d604-4c4e-8754-d1bdb0ab50a3 service neutron] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:03.520798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23276de9-d604-4c4e-8754-d1bdb0ab50a3 service neutron] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:03.647486 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4107573b-e7a4-4b30-a19a-54864b4dcea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:03.648726 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107573b-e7a4-4b30-a19a-54864b4dcea2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:03.649248 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107573b-e7a4-4b30-a19a-54864b4dcea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:03.649721 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107573b-e7a4-4b30-a19a-54864b4dcea2 admin admin] PATH_INFO: `/v3/users/d60b20faff2f4d658219096f3e3e4cb7` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:03.651483 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4107573b-e7a4-4b30-a19a-54864b4dcea2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d60b20faff2f4d658219096f3e3e4cb7)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:03.654476 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4107573b-e7a4-4b30-a19a-54864b4dcea2 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:03.728832 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-4107573b-e7a4-4b30-a19a-54864b4dcea2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd60b20faff2f4d658219096f3e3e4cb7'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:03.762944 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-4107573b-e7a4-4b30-a19a-54864b4dcea2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd60b20faff2f4d658219096f3e3e4cb7'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:03.785815 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-4107573b-e7a4-4b30-a19a-54864b4dcea2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd60b20faff2f4d658219096f3e3e4cb7'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:03.799109 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-4107573b-e7a4-4b30-a19a-54864b4dcea2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd60b20faff2f4d658219096f3e3e4cb7'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:03.800491 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-4107573b-e7a4-4b30-a19a-54864b4dcea2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd60b20faff2f4d658219096f3e3e4cb7'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:03.808296 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 926/1837] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:57:03 2022] DELETE /identity/v3/users/d60b20faff2f4d658219096f3e3e4cb7 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:03.855003 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82928bc0-1398-45b6-80fc-4c2a36998e2a admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:03.856529 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-82928bc0-1398-45b6-80fc-4c2a36998e2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:03.922271 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82928bc0-1398-45b6-80fc-4c2a36998e2a admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:04.000777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8f595b-0f42-4079-9e15-5762221476b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:04.001644 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f595b-0f42-4079-9e15-5762221476b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:04.001951 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f595b-0f42-4079-9e15-5762221476b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:04.002221 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f595b-0f42-4079-9e15-5762221476b1 admin admin] PATH_INFO: `/v3/projects/b2b6765d13ed46b985141a686ef8409d` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:04.003455 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8f595b-0f42-4079-9e15-5762221476b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2b6765d13ed46b985141a686ef8409d)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:04.005471 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8f595b-0f42-4079-9e15-5762221476b1 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:04.055756 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-ea8f595b-0f42-4079-9e15-5762221476b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2b6765d13ed46b985141a686ef8409d', 'request_id': 'req-ea8f595b-0f42-4079-9e15-5762221476b1'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:04.062093 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-ea8f595b-0f42-4079-9e15-5762221476b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2b6765d13ed46b985141a686ef8409d', 'request_id': 'req-ea8f595b-0f42-4079-9e15-5762221476b1'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:04.070853 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-ea8f595b-0f42-4079-9e15-5762221476b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2b6765d13ed46b985141a686ef8409d', 'request_id': 'req-ea8f595b-0f42-4079-9e15-5762221476b1'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:04.082646 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-ea8f595b-0f42-4079-9e15-5762221476b1 admin admin] The token cache is being invalidate because project b2b6765d13ed46b985141a686ef8409d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:57:04.082849 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-ea8f595b-0f42-4079-9e15-5762221476b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:04.083536 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-ea8f595b-0f42-4079-9e15-5762221476b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:04.085122 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 912/1838] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:57:03 2022] DELETE /identity/v3/projects/b2b6765d13ed46b985141a686ef8409d => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:04.089158 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4107573b-e7a4-4b30-a19a-54864b4dcea2 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:04.090782 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-4107573b-e7a4-4b30-a19a-54864b4dcea2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:04.105160 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4107573b-e7a4-4b30-a19a-54864b4dcea2 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:04.124128 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:04.174725 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6af638c-57f2-430a-9f76-bb0c332a4155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:04.175722 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af638c-57f2-430a-9f76-bb0c332a4155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:04.176004 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af638c-57f2-430a-9f76-bb0c332a4155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:04.176499 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af638c-57f2-430a-9f76-bb0c332a4155 admin admin] PATH_INFO: `/v3/users/4949d25a48a04780857999b19b3f8b65` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:04.177722 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6af638c-57f2-430a-9f76-bb0c332a4155 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4949d25a48a04780857999b19b3f8b65)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:04.179708 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6af638c-57f2-430a-9f76-bb0c332a4155 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:04.233460 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-e6af638c-57f2-430a-9f76-bb0c332a4155 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4949d25a48a04780857999b19b3f8b65'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:04.247746 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-e6af638c-57f2-430a-9f76-bb0c332a4155 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4949d25a48a04780857999b19b3f8b65'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:04.257115 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-e6af638c-57f2-430a-9f76-bb0c332a4155 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4949d25a48a04780857999b19b3f8b65'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:04.271576 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-e6af638c-57f2-430a-9f76-bb0c332a4155 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4949d25a48a04780857999b19b3f8b65'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:04.272797 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-e6af638c-57f2-430a-9f76-bb0c332a4155 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4949d25a48a04780857999b19b3f8b65'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:04.285395 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 927/1839] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:57:04 2022] DELETE /identity/v3/users/4949d25a48a04780857999b19b3f8b65 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:04.523037 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8f595b-0f42-4079-9e15-5762221476b1 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:04.527603 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-ea8f595b-0f42-4079-9e15-5762221476b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:04.539931 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8f595b-0f42-4079-9e15-5762221476b1 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:04.608405 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1ccc7a-5ef9-48df-85ee-c747e10cbbdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:04.609288 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ccc7a-5ef9-48df-85ee-c747e10cbbdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:04.609570 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ccc7a-5ef9-48df-85ee-c747e10cbbdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:04.609846 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ccc7a-5ef9-48df-85ee-c747e10cbbdf admin admin] PATH_INFO: `/v3/projects/305f45bf58e84e8f90a9b99fcfa26477` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:04.611040 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1ccc7a-5ef9-48df-85ee-c747e10cbbdf admin admin] RBAC: Authorizing `identity:delete_project(project_id=305f45bf58e84e8f90a9b99fcfa26477)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:04.613037 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1ccc7a-5ef9-48df-85ee-c747e10cbbdf admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:04.653863 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:04.675042 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-8b1ccc7a-5ef9-48df-85ee-c747e10cbbdf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '305f45bf58e84e8f90a9b99fcfa26477', 'request_id': 'req-8b1ccc7a-5ef9-48df-85ee-c747e10cbbdf'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:04.677151 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6af638c-57f2-430a-9f76-bb0c332a4155 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:04.678706 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-e6af638c-57f2-430a-9f76-bb0c332a4155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:04.683983 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-8b1ccc7a-5ef9-48df-85ee-c747e10cbbdf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '305f45bf58e84e8f90a9b99fcfa26477', 'request_id': 'req-8b1ccc7a-5ef9-48df-85ee-c747e10cbbdf'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:04.694386 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-8b1ccc7a-5ef9-48df-85ee-c747e10cbbdf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '305f45bf58e84e8f90a9b99fcfa26477', 'request_id': 'req-8b1ccc7a-5ef9-48df-85ee-c747e10cbbdf'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:04.706234 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-8b1ccc7a-5ef9-48df-85ee-c747e10cbbdf admin admin] The token cache is being invalidate because project 305f45bf58e84e8f90a9b99fcfa26477 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:57:04.706705 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-8b1ccc7a-5ef9-48df-85ee-c747e10cbbdf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:04.707832 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-8b1ccc7a-5ef9-48df-85ee-c747e10cbbdf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:04.709379 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6af638c-57f2-430a-9f76-bb0c332a4155 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:04.709762 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 913/1840] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:57:04 2022] DELETE /identity/v3/projects/305f45bf58e84e8f90a9b99fcfa26477 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:04.714809 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc90b8-19c9-463a-b6d4-5ad1201f1815 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:04.715163 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc90b8-19c9-463a-b6d4-5ad1201f1815 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:04.715645 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc90b8-19c9-463a-b6d4-5ad1201f1815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:04.738137 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-4ecc90b8-19c9-463a-b6d4-5ad1201f1815 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:04.739857 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-4ecc90b8-19c9-463a-b6d4-5ad1201f1815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:04.789473 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6d0804-1b0f-4718-bb2e-4edcd49d33e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:04.790684 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d0804-1b0f-4718-bb2e-4edcd49d33e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:04.790970 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d0804-1b0f-4718-bb2e-4edcd49d33e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:04.791591 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6d0804-1b0f-4718-bb2e-4edcd49d33e7 admin admin] PATH_INFO: `/v3/projects/6305c5cedc0a4f51accfaae2ffe8e3d1` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:04.792494 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6d0804-1b0f-4718-bb2e-4edcd49d33e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6305c5cedc0a4f51accfaae2ffe8e3d1)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:04.794395 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6d0804-1b0f-4718-bb2e-4edcd49d33e7 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:04.806695 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 914/1841] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 01:57:04.816999 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a51a7-3599-43af-94bf-5b6d175d3520 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:04.817285 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a51a7-3599-43af-94bf-5b6d175d3520 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:04.817566 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a51a7-3599-43af-94bf-5b6d175d3520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:04.837673 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-d11a51a7-3599-43af-94bf-5b6d175d3520 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:04.839044 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-d11a51a7-3599-43af-94bf-5b6d175d3520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:04.841691 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:04.845572 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-be6d0804-1b0f-4718-bb2e-4edcd49d33e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6305c5cedc0a4f51accfaae2ffe8e3d1', 'request_id': 'req-be6d0804-1b0f-4718-bb2e-4edcd49d33e7'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:04.853913 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-be6d0804-1b0f-4718-bb2e-4edcd49d33e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6305c5cedc0a4f51accfaae2ffe8e3d1', 'request_id': 'req-be6d0804-1b0f-4718-bb2e-4edcd49d33e7'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:04.863099 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-be6d0804-1b0f-4718-bb2e-4edcd49d33e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6305c5cedc0a4f51accfaae2ffe8e3d1', 'request_id': 'req-be6d0804-1b0f-4718-bb2e-4edcd49d33e7'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:04.874258 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-be6d0804-1b0f-4718-bb2e-4edcd49d33e7 admin admin] The token cache is being invalidate because project 6305c5cedc0a4f51accfaae2ffe8e3d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:57:04.874543 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-be6d0804-1b0f-4718-bb2e-4edcd49d33e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:04.875185 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-be6d0804-1b0f-4718-bb2e-4edcd49d33e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:04.878869 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 928/1842] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:57:04 2022] DELETE /identity/v3/projects/6305c5cedc0a4f51accfaae2ffe8e3d1 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:04.883920 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bafb23-5912-4654-b411-30bcc1c17dbe None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:04.884207 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bafb23-5912-4654-b411-30bcc1c17dbe None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:04.884524 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bafb23-5912-4654-b411-30bcc1c17dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:04.902531 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-91bafb23-5912-4654-b411-30bcc1c17dbe None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:04.903767 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-91bafb23-5912-4654-b411-30bcc1c17dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:04.904217 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 915/1843] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 01:57:04.908082 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11a51a7-3599-43af-94bf-5b6d175d3520 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:04.912729 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 929/1844] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:04.919362 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11a51a7-3599-43af-94bf-5b6d175d3520 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:04.927356 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d7fcd-d631-4c28-80f1-447cbcd71ba4 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:04.928012 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d7fcd-d631-4c28-80f1-447cbcd71ba4 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:04.928012 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d7fcd-d631-4c28-80f1-447cbcd71ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:04.931302 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd0bf4-b1dc-4414-9c01-d60da51da40f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:04.932095 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd0bf4-b1dc-4414-9c01-d60da51da40f admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:04.932439 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd0bf4-b1dc-4414-9c01-d60da51da40f admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:04.932780 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd0bf4-b1dc-4414-9c01-d60da51da40f admin admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:04.934058 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fd0bf4-b1dc-4414-9c01-d60da51da40f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:57:04.934949 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fd0bf4-b1dc-4414-9c01-d60da51da40f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:04.937115 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fd0bf4-b1dc-4414-9c01-d60da51da40f admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:04.957108 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 916/1845] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 01:57:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:57:04.958923 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd0bf4-b1dc-4414-9c01-d60da51da40f admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:04.961159 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-d89d7fcd-d631-4c28-80f1-447cbcd71ba4 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:04.962378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-d89d7fcd-d631-4c28-80f1-447cbcd71ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:04.983040 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd0bf4-b1dc-4414-9c01-d60da51da40f admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:04.984973 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 930/1846] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 01:57:04.989144 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d7fcd-d631-4c28-80f1-447cbcd71ba4 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:04.989540 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a42fb67-bd5a-4bde-afbd-563a687a4722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:04.990451 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a42fb67-bd5a-4bde-afbd-563a687a4722 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:04.991571 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a42fb67-bd5a-4bde-afbd-563a687a4722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:04.992068 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a42fb67-bd5a-4bde-afbd-563a687a4722 admin admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:04.992972 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a42fb67-bd5a-4bde-afbd-563a687a4722 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:04.995303 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a42fb67-bd5a-4bde-afbd-563a687a4722 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:05.005888 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d7fcd-d631-4c28-80f1-447cbcd71ba4 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:05.011790 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c36e58-27a9-47b2-a53b-81bf9bdadeca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:05.012584 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c36e58-27a9-47b2-a53b-81bf9bdadeca admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:05.012961 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c36e58-27a9-47b2-a53b-81bf9bdadeca admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:05.013333 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c36e58-27a9-47b2-a53b-81bf9bdadeca admin admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:05.014176 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c36e58-27a9-47b2-a53b-81bf9bdadeca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:57:05.014577 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c36e58-27a9-47b2-a53b-81bf9bdadeca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:05.016637 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c36e58-27a9-47b2-a53b-81bf9bdadeca admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:05.037198 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 931/1847] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 01:57:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:57:05.041571 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c36e58-27a9-47b2-a53b-81bf9bdadeca admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:05.057228 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 917/1848] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:57:04 2022] POST /identity/v3/projects => generated 399 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:57:05.057873 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c36e58-27a9-47b2-a53b-81bf9bdadeca admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:05.061457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a42fb67-bd5a-4bde-afbd-563a687a4722 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:05.097303 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:05.140222 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a42fb67-bd5a-4bde-afbd-563a687a4722 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:05.146370 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957509b2-0bd1-4dc9-aa27-e0bf63b4962e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:05.147871 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957509b2-0bd1-4dc9-aa27-e0bf63b4962e admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:05.148139 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d346a6-53e9-47e3-874f-e75d094b4337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:05.148714 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957509b2-0bd1-4dc9-aa27-e0bf63b4962e admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:05.148843 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d346a6-53e9-47e3-874f-e75d094b4337 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:05.149081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d346a6-53e9-47e3-874f-e75d094b4337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:05.149361 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d346a6-53e9-47e3-874f-e75d094b4337 admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:05.149536 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957509b2-0bd1-4dc9-aa27-e0bf63b4962e admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:05.150060 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d346a6-53e9-47e3-874f-e75d094b4337 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:05.150352 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957509b2-0bd1-4dc9-aa27-e0bf63b4962e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:05.152131 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d346a6-53e9-47e3-874f-e75d094b4337 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:05.152427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957509b2-0bd1-4dc9-aa27-e0bf63b4962e admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:05.178519 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 918/1849] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:57:05 2022] POST /identity/v3/users => generated 300 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 01:57:05.183336 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957509b2-0bd1-4dc9-aa27-e0bf63b4962e admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:05.188023 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 932/1850] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:57:05 2022] POST /identity/v3/projects => generated 379 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 01:57:05.192754 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d346a6-53e9-47e3-874f-e75d094b4337 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:05.195136 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957509b2-0bd1-4dc9-aa27-e0bf63b4962e admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:05.267878 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac0050-f364-4f0d-ba50-242bb3496d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:05.268695 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac0050-f364-4f0d-ba50-242bb3496d9b admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:05.269167 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac0050-f364-4f0d-ba50-242bb3496d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:05.269419 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac0050-f364-4f0d-ba50-242bb3496d9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:05.270166 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faac0050-f364-4f0d-ba50-242bb3496d9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:05.271936 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faac0050-f364-4f0d-ba50-242bb3496d9b admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:05.279351 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d346a6-53e9-47e3-874f-e75d094b4337 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:05.287338 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 919/1851] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 01:57:05.288116 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37c9d0-6faf-405b-afe8-21de855175bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:05.288838 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac0050-f364-4f0d-ba50-242bb3496d9b admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:05.289001 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37c9d0-6faf-405b-afe8-21de855175bb admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:05.289374 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37c9d0-6faf-405b-afe8-21de855175bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:05.289755 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37c9d0-6faf-405b-afe8-21de855175bb admin admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:05.290687 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c37c9d0-6faf-405b-afe8-21de855175bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:05.292955 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c37c9d0-6faf-405b-afe8-21de855175bb admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:05.305707 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac0050-f364-4f0d-ba50-242bb3496d9b admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:05.311365 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775a2392-df9f-415a-a3c2-62076a8999ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:05.312197 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a2392-df9f-415a-a3c2-62076a8999ad admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:05.312545 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a2392-df9f-415a-a3c2-62076a8999ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:05.312926 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a2392-df9f-415a-a3c2-62076a8999ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:05.313852 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775a2392-df9f-415a-a3c2-62076a8999ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:05.315677 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775a2392-df9f-415a-a3c2-62076a8999ad admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:05.320767 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 933/1852] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:57:05 2022] POST /identity/v3/users => generated 290 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 01:57:05.321859 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37c9d0-6faf-405b-afe8-21de855175bb admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:05.331309 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 920/1853] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 01:57:05.337660 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775a2392-df9f-415a-a3c2-62076a8999ad admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:05.340991 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37c9d0-6faf-405b-afe8-21de855175bb admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:05.350182 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775a2392-df9f-415a-a3c2-62076a8999ad admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:05.355672 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ca868-ba0d-4e73-9e1b-debd59e0c140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:05.356647 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ca868-ba0d-4e73-9e1b-debd59e0c140 admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:05.356993 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ca868-ba0d-4e73-9e1b-debd59e0c140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:05.357342 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ca868-ba0d-4e73-9e1b-debd59e0c140 admin admin] PATH_INFO: `/v3/projects/3142b780b5354cd0ac4e3a88d4233562/users/5d2a25e6872641b6824b00633b7d2117/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:05.364883 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b13d2-40cd-4f2b-abca-15e34b743b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:05.365574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b13d2-40cd-4f2b-abca-15e34b743b2c admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:05.365860 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b13d2-40cd-4f2b-abca-15e34b743b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:05.366129 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b13d2-40cd-4f2b-abca-15e34b743b2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:05.366879 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2b13d2-40cd-4f2b-abca-15e34b743b2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:05.368605 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2b13d2-40cd-4f2b-abca-15e34b743b2c admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:05.384423 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 934/1854] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:57:05.388866 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b13d2-40cd-4f2b-abca-15e34b743b2c admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:05.401866 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6ca868-ba0d-4e73-9e1b-debd59e0c140 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3142b780b5354cd0ac4e3a88d4233562, user_id=5d2a25e6872641b6824b00633b7d2117, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:05.402157 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b13d2-40cd-4f2b-abca-15e34b743b2c admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:05.406199 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6ca868-ba0d-4e73-9e1b-debd59e0c140 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:05.409349 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f59b99-0ee8-4627-9c5f-9d0954cf077f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:05.410281 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f59b99-0ee8-4627-9c5f-9d0954cf077f admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:05.410713 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f59b99-0ee8-4627-9c5f-9d0954cf077f admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:05.410924 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f59b99-0ee8-4627-9c5f-9d0954cf077f admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:05.411995 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f59b99-0ee8-4627-9c5f-9d0954cf077f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:05.414112 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f59b99-0ee8-4627-9c5f-9d0954cf077f admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:05.422877 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:05.433485 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 921/1855] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:57:05 2022] PUT /identity/v3/projects/3142b780b5354cd0ac4e3a88d4233562/users/5d2a25e6872641b6824b00633b7d2117/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:05.443605 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 935/1856] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:57:05.445476 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4af35-4641-4252-97cc-a5830b466508 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:05.445643 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f59b99-0ee8-4627-9c5f-9d0954cf077f admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:05.446071 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4af35-4641-4252-97cc-a5830b466508 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:05.446565 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed4af35-4641-4252-97cc-a5830b466508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:05.457772 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f59b99-0ee8-4627-9c5f-9d0954cf077f admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:05.495980 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-7ed4af35-4641-4252-97cc-a5830b466508 None None] MFA Rules not processed for user `5d2a25e6872641b6824b00633b7d2117`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:05.495980 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-7ed4af35-4641-4252-97cc-a5830b466508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:05.529758 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83abf72-b900-48ef-ba92-7f933a525981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:05.530784 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83abf72-b900-48ef-ba92-7f933a525981 admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:05.531131 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83abf72-b900-48ef-ba92-7f933a525981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:05.531872 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83abf72-b900-48ef-ba92-7f933a525981 admin admin] PATH_INFO: `/v3/projects/21ee263f4d7e4f7c94d4f521e78d8480/users/7b094d1fb1324b65b581386f7af619b7/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:05.550673 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83abf72-b900-48ef-ba92-7f933a525981 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21ee263f4d7e4f7c94d4f521e78d8480, user_id=7b094d1fb1324b65b581386f7af619b7, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:05.553792 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83abf72-b900-48ef-ba92-7f933a525981 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:05.567550 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 922/1857] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:05 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:05.572185 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed4af35-4641-4252-97cc-a5830b466508 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:05.575592 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 936/1858] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:57:05 2022] PUT /identity/v3/projects/21ee263f4d7e4f7c94d4f521e78d8480/users/7b094d1fb1324b65b581386f7af619b7/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:05.580446 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:05.586300 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed4af35-4641-4252-97cc-a5830b466508 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:05.587506 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2255400-7d8b-4f3d-a491-ac28ffa4ecd2 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:05.587811 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2255400-7d8b-4f3d-a491-ac28ffa4ecd2 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:05.588073 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2255400-7d8b-4f3d-a491-ac28ffa4ecd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:05.619909 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-c2255400-7d8b-4f3d-a491-ac28ffa4ecd2 None None] MFA Rules not processed for user `7b094d1fb1324b65b581386f7af619b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:05.621175 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-c2255400-7d8b-4f3d-a491-ac28ffa4ecd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:05.668019 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d801d7-fe7e-496d-b2bb-7c8c28dc51ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:05.668683 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d801d7-fe7e-496d-b2bb-7c8c28dc51ad admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:05.668963 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d801d7-fe7e-496d-b2bb-7c8c28dc51ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:05.669241 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d801d7-fe7e-496d-b2bb-7c8c28dc51ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:05.669936 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d801d7-fe7e-496d-b2bb-7c8c28dc51ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:05.671952 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d801d7-fe7e-496d-b2bb-7c8c28dc51ad admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:05.694225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 937/1859] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:05.698069 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 923/1860] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:57:05 2022] POST /identity/v3/projects => generated 401 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:57:05.698868 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2255400-7d8b-4f3d-a491-ac28ffa4ecd2 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:05.701651 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d801d7-fe7e-496d-b2bb-7c8c28dc51ad admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:05.709781 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2255400-7d8b-4f3d-a491-ac28ffa4ecd2 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:05.762446 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d801d7-fe7e-496d-b2bb-7c8c28dc51ad admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:05.764083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc375ebd-9210-4e11-a8e3-2fd67ac05d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:05.764722 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc375ebd-9210-4e11-a8e3-2fd67ac05d3c admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:05.764997 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc375ebd-9210-4e11-a8e3-2fd67ac05d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:05.765265 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc375ebd-9210-4e11-a8e3-2fd67ac05d3c admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:05.765955 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc375ebd-9210-4e11-a8e3-2fd67ac05d3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:05.767373 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c015fe-f68a-408a-8850-a6e716f67498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:05.767909 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc375ebd-9210-4e11-a8e3-2fd67ac05d3c admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:05.768164 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c015fe-f68a-408a-8850-a6e716f67498 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:05.768449 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c015fe-f68a-408a-8850-a6e716f67498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:05.768726 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c015fe-f68a-408a-8850-a6e716f67498 admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:05.769426 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c015fe-f68a-408a-8850-a6e716f67498 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:05.771416 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c015fe-f68a-408a-8850-a6e716f67498 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:05.782486 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:05.795094 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 924/1861] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:57:05 2022] POST /identity/v3/users => generated 307 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 01:57:05.798014 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 938/1862] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:57:05 2022] POST /identity/v3/projects => generated 379 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 01:57:05.799622 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c015fe-f68a-408a-8850-a6e716f67498 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:05.802136 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc375ebd-9210-4e11-a8e3-2fd67ac05d3c admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:05.810462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c015fe-f68a-408a-8850-a6e716f67498 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:05.868743 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc375ebd-9210-4e11-a8e3-2fd67ac05d3c admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:05.873471 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5a0c8a-8328-441e-86df-629ae4923d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:05.874142 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a0c8a-8328-441e-86df-629ae4923d3c admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:05.874421 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a0c8a-8328-441e-86df-629ae4923d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:05.874693 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a0c8a-8328-441e-86df-629ae4923d3c admin admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:05.875416 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5a0c8a-8328-441e-86df-629ae4923d3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:05.876286 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ed3d6-6458-4fbd-8ada-71f0f423e573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:05.877179 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ed3d6-6458-4fbd-8ada-71f0f423e573 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:05.877345 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5a0c8a-8328-441e-86df-629ae4923d3c admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:05.877567 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ed3d6-6458-4fbd-8ada-71f0f423e573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:05.877855 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ed3d6-6458-4fbd-8ada-71f0f423e573 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:05.878694 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796ed3d6-6458-4fbd-8ada-71f0f423e573 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:05.880443 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796ed3d6-6458-4fbd-8ada-71f0f423e573 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:05.892356 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 925/1863] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:57:05.897205 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ed3d6-6458-4fbd-8ada-71f0f423e573 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:05.899917 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 939/1864] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:57:05 2022] POST /identity/v3/users => generated 296 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:57:05.904674 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5a0c8a-8328-441e-86df-629ae4923d3c admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:05.908280 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ed3d6-6458-4fbd-8ada-71f0f423e573 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:05.917279 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5a0c8a-8328-441e-86df-629ae4923d3c admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:05.921871 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad27a6f5-2449-4114-af38-f65ef57af177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:05.922283 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf3838-214d-462b-848e-d4e4bef34dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:05.922773 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27a6f5-2449-4114-af38-f65ef57af177 admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:05.923083 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27a6f5-2449-4114-af38-f65ef57af177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:05.923244 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf3838-214d-462b-848e-d4e4bef34dbe admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:05.923433 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27a6f5-2449-4114-af38-f65ef57af177 admin admin] PATH_INFO: `/v3/projects/1fa87b8ef18a411b83618c37319756c3/users/4d8b027d22a1435ea591392f5bee8453/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:05.923599 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf3838-214d-462b-848e-d4e4bef34dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:05.923950 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf3838-214d-462b-848e-d4e4bef34dbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:05.925106 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbf3838-214d-462b-848e-d4e4bef34dbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:05.927059 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbf3838-214d-462b-848e-d4e4bef34dbe admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:05.936707 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:05.937495 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 940/1865] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 01:57:05.941367 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad27a6f5-2449-4114-af38-f65ef57af177 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fa87b8ef18a411b83618c37319756c3, user_id=4d8b027d22a1435ea591392f5bee8453, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:05.941845 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf3838-214d-462b-848e-d4e4bef34dbe admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:05.944399 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad27a6f5-2449-4114-af38-f65ef57af177 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:05.951924 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf3838-214d-462b-848e-d4e4bef34dbe admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:05.957061 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890a9508-7aaa-4ca1-9d48-982428ea86de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:05.957952 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a9508-7aaa-4ca1-9d48-982428ea86de admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:05.958262 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a9508-7aaa-4ca1-9d48-982428ea86de admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:05.958541 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a9508-7aaa-4ca1-9d48-982428ea86de admin admin] PATH_INFO: `/v3/projects/f0592e3e6647415f98b273e1d8833887/users/ac63f3f2a81945e7b7af62848d179da4/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:05.959488 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 926/1866] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:57:05 2022] PUT /identity/v3/projects/1fa87b8ef18a411b83618c37319756c3/users/4d8b027d22a1435ea591392f5bee8453/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:05.963669 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad27a6f5-2449-4114-af38-f65ef57af177 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:05.974094 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890a9508-7aaa-4ca1-9d48-982428ea86de admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0592e3e6647415f98b273e1d8833887, user_id=ac63f3f2a81945e7b7af62848d179da4, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:05.976850 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890a9508-7aaa-4ca1-9d48-982428ea86de admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:05.996375 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 941/1867] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:57:05 2022] PUT /identity/v3/projects/f0592e3e6647415f98b273e1d8833887/users/ac63f3f2a81945e7b7af62848d179da4/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 01:57:05.997766 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890a9508-7aaa-4ca1-9d48-982428ea86de admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:06.031757 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad27a6f5-2449-4114-af38-f65ef57af177 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:06.064322 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890a9508-7aaa-4ca1-9d48-982428ea86de admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:06.069557 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab9af0-09c8-4c83-b09c-42ba0d6fda82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:06.070314 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab9af0-09c8-4c83-b09c-42ba0d6fda82 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:06.070658 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab9af0-09c8-4c83-b09c-42ba0d6fda82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:06.071106 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab9af0-09c8-4c83-b09c-42ba0d6fda82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:06.071928 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aab9af0-09c8-4c83-b09c-42ba0d6fda82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:06.073640 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aab9af0-09c8-4c83-b09c-42ba0d6fda82 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:06.084277 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 942/1868] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 01:57:06.088419 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab9af0-09c8-4c83-b09c-42ba0d6fda82 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:06.096702 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbadb130-ff6e-4b76-bbd8-26082d261850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:06.097494 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbadb130-ff6e-4b76-bbd8-26082d261850 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:06.097743 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbadb130-ff6e-4b76-bbd8-26082d261850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:06.098097 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbadb130-ff6e-4b76-bbd8-26082d261850 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:06.098836 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbadb130-ff6e-4b76-bbd8-26082d261850 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:06.100485 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbadb130-ff6e-4b76-bbd8-26082d261850 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:06.100761 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab9af0-09c8-4c83-b09c-42ba0d6fda82 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:06.105459 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d9860a-d039-4465-8bcf-53570049c140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:06.106324 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d9860a-d039-4465-8bcf-53570049c140 admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:06.106597 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d9860a-d039-4465-8bcf-53570049c140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:06.106867 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d9860a-d039-4465-8bcf-53570049c140 admin admin] PATH_INFO: `/v3/projects/f0592e3e6647415f98b273e1d8833887/users/ac63f3f2a81945e7b7af62848d179da4/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:06.108946 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d9860a-d039-4465-8bcf-53570049c140 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0592e3e6647415f98b273e1d8833887, user_id=ac63f3f2a81945e7b7af62848d179da4, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:06.114102 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 927/1869] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 01:57:06.114237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d9860a-d039-4465-8bcf-53570049c140 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:06.116755 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbadb130-ff6e-4b76-bbd8-26082d261850 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:06.122455 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 943/1870] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:57:06 2022] PUT /identity/v3/projects/f0592e3e6647415f98b273e1d8833887/users/ac63f3f2a81945e7b7af62848d179da4/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 01:57:06.126361 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbadb130-ff6e-4b76-bbd8-26082d261850 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:06.132392 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db4676-1249-448c-a418-03d8cf20d94f None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:06.132688 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db4676-1249-448c-a418-03d8cf20d94f None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:06.133000 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db4676-1249-448c-a418-03d8cf20d94f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:06.160651 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-b0db4676-1249-448c-a418-03d8cf20d94f None None] MFA Rules not processed for user `ac63f3f2a81945e7b7af62848d179da4`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:06.161909 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-b0db4676-1249-448c-a418-03d8cf20d94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:06.183956 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:06.190223 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff940b00-f33f-46c4-b9d2-0d47052e00ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:06.191336 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff940b00-f33f-46c4-b9d2-0d47052e00ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:06.191614 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff940b00-f33f-46c4-b9d2-0d47052e00ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:06.191889 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff940b00-f33f-46c4-b9d2-0d47052e00ed admin admin] PATH_INFO: `/v3/projects/1fa87b8ef18a411b83618c37319756c3/users/4d8b027d22a1435ea591392f5bee8453/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:06.193946 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff940b00-f33f-46c4-b9d2-0d47052e00ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fa87b8ef18a411b83618c37319756c3, user_id=4d8b027d22a1435ea591392f5bee8453, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:06.196800 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff940b00-f33f-46c4-b9d2-0d47052e00ed admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:06.211876 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 928/1871] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:57:06 2022] PUT /identity/v3/projects/1fa87b8ef18a411b83618c37319756c3/users/4d8b027d22a1435ea591392f5bee8453/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:06.218985 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba71382-50ce-49dc-b51b-7121f1ba7a40 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:06.219484 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba71382-50ce-49dc-b51b-7121f1ba7a40 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:06.219907 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba71382-50ce-49dc-b51b-7121f1ba7a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:06.243432 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 944/1872] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:06 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 01:57:06.247415 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-5ba71382-50ce-49dc-b51b-7121f1ba7a40 None None] MFA Rules not processed for user `4d8b027d22a1435ea591392f5bee8453`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:06.248721 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-5ba71382-50ce-49dc-b51b-7121f1ba7a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:06.256865 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db4676-1249-448c-a418-03d8cf20d94f None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:06.258751 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-b0db4676-1249-448c-a418-03d8cf20d94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:06.274007 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:06.324321 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db4676-1249-448c-a418-03d8cf20d94f None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:06.332476 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 929/1873] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:06 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:06.343095 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba71382-50ce-49dc-b51b-7121f1ba7a40 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:06.344571 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-5ba71382-50ce-49dc-b51b-7121f1ba7a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:06.371502 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6684d49-476a-40c0-9f47-92c26ad52232 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:06.372242 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6684d49-476a-40c0-9f47-92c26ad52232 service cinder] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:06.372518 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6684d49-476a-40c0-9f47-92c26ad52232 service cinder] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:06.372868 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6684d49-476a-40c0-9f47-92c26ad52232 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:06.401350 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba71382-50ce-49dc-b51b-7121f1ba7a40 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:06.405017 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a5e1ac-23bc-4674-ae8f-42666d241fae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:06.405667 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a5e1ac-23bc-4674-ae8f-42666d241fae service cinder] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:06.405949 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a5e1ac-23bc-4674-ae8f-42666d241fae service cinder] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:06.406219 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a5e1ac-23bc-4674-ae8f-42666d241fae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:06.415749 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a5e1ac-23bc-4674-ae8f-42666d241fae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:06.417215 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a5e1ac-23bc-4674-ae8f-42666d241fae service cinder] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:06.426862 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 930/1874] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:57:06 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 01:57:06.436191 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6684d49-476a-40c0-9f47-92c26ad52232 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:06.437643 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6684d49-476a-40c0-9f47-92c26ad52232 service cinder] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:06.447541 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 945/1875] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:57:06 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 01:57:06.504432 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a5e1ac-23bc-4674-ae8f-42666d241fae service cinder] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:06.513195 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a5e1ac-23bc-4674-ae8f-42666d241fae service cinder] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:06.517191 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d523e81-03d2-4529-97c2-91eee18ed03d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:06.517917 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d523e81-03d2-4529-97c2-91eee18ed03d service cinder] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:06.518188 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d523e81-03d2-4529-97c2-91eee18ed03d service cinder] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:06.518461 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d523e81-03d2-4529-97c2-91eee18ed03d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:06.579430 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d523e81-03d2-4529-97c2-91eee18ed03d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:06.580929 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d523e81-03d2-4529-97c2-91eee18ed03d service cinder] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:06.595878 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 931/1876] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:57:06 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:57:06.896547 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6684d49-476a-40c0-9f47-92c26ad52232 service cinder] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:06.907005 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6684d49-476a-40c0-9f47-92c26ad52232 service cinder] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:06.914672 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5fdeb6-eb78-408c-81e1-45f0b1b19986 tempest-VolumeQuotaClassesTest-918343840 tempest-VolumeQuotaClassesTest-918343840-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac63f3f2a81945e7b7af62848d179da4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0592e3e6647415f98b273e1d8833887', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:06.931604 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5fdeb6-eb78-408c-81e1-45f0b1b19986 tempest-VolumeQuotaClassesTest-918343840 tempest-VolumeQuotaClassesTest-918343840-project-admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:06.931604 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5fdeb6-eb78-408c-81e1-45f0b1b19986 tempest-VolumeQuotaClassesTest-918343840 tempest-VolumeQuotaClassesTest-918343840-project-admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:06.931604 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5fdeb6-eb78-408c-81e1-45f0b1b19986 tempest-VolumeQuotaClassesTest-918343840 tempest-VolumeQuotaClassesTest-918343840-project-admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:06.931604 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5fdeb6-eb78-408c-81e1-45f0b1b19986 tempest-VolumeQuotaClassesTest-918343840 tempest-VolumeQuotaClassesTest-918343840-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:57:06.931604 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5fdeb6-eb78-408c-81e1-45f0b1b19986 tempest-VolumeQuotaClassesTest-918343840 tempest-VolumeQuotaClassesTest-918343840-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:06.932330 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5fdeb6-eb78-408c-81e1-45f0b1b19986 tempest-VolumeQuotaClassesTest-918343840 tempest-VolumeQuotaClassesTest-918343840-project-admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:06.947464 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 946/1877] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 01:57:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:57:06.951555 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d523e81-03d2-4529-97c2-91eee18ed03d service cinder] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:06.968368 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d523e81-03d2-4529-97c2-91eee18ed03d service cinder] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:06.973994 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793d63ed-8418-4f65-bebe-96c71249cb99 tempest-VolumeQuotaClassesTest-918343840 tempest-VolumeQuotaClassesTest-918343840-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac63f3f2a81945e7b7af62848d179da4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0592e3e6647415f98b273e1d8833887', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:06.974787 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d63ed-8418-4f65-bebe-96c71249cb99 tempest-VolumeQuotaClassesTest-918343840 tempest-VolumeQuotaClassesTest-918343840-project-admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:06.975176 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d63ed-8418-4f65-bebe-96c71249cb99 tempest-VolumeQuotaClassesTest-918343840 tempest-VolumeQuotaClassesTest-918343840-project-admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:06.975582 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d63ed-8418-4f65-bebe-96c71249cb99 tempest-VolumeQuotaClassesTest-918343840 tempest-VolumeQuotaClassesTest-918343840-project-admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:06.976497 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793d63ed-8418-4f65-bebe-96c71249cb99 tempest-VolumeQuotaClassesTest-918343840 tempest-VolumeQuotaClassesTest-918343840-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:06.978766 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793d63ed-8418-4f65-bebe-96c71249cb99 tempest-VolumeQuotaClassesTest-918343840 tempest-VolumeQuotaClassesTest-918343840-project-admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:07.010107 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 932/1878] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:57:06 2022] POST /identity/v3/projects => generated 355 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:57:07.014074 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5fdeb6-eb78-408c-81e1-45f0b1b19986 tempest-VolumeQuotaClassesTest-918343840 tempest-VolumeQuotaClassesTest-918343840-project-admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:07.026946 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5fdeb6-eb78-408c-81e1-45f0b1b19986 tempest-VolumeQuotaClassesTest-918343840 tempest-VolumeQuotaClassesTest-918343840-project-admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:07.035912 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:07.087766 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b4e1e0-d061-447d-89eb-83aa568c1a3f tempest-VolumeQuotaClassesTest-918343840 tempest-VolumeQuotaClassesTest-918343840-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac63f3f2a81945e7b7af62848d179da4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0592e3e6647415f98b273e1d8833887', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:07.088429 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4e1e0-d061-447d-89eb-83aa568c1a3f tempest-VolumeQuotaClassesTest-918343840 tempest-VolumeQuotaClassesTest-918343840-project-admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:07.088698 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4e1e0-d061-447d-89eb-83aa568c1a3f tempest-VolumeQuotaClassesTest-918343840 tempest-VolumeQuotaClassesTest-918343840-project-admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:07.089001 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4e1e0-d061-447d-89eb-83aa568c1a3f tempest-VolumeQuotaClassesTest-918343840 tempest-VolumeQuotaClassesTest-918343840-project-admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:07.089911 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b4e1e0-d061-447d-89eb-83aa568c1a3f tempest-VolumeQuotaClassesTest-918343840 tempest-VolumeQuotaClassesTest-918343840-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:57:07.090216 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b4e1e0-d061-447d-89eb-83aa568c1a3f tempest-VolumeQuotaClassesTest-918343840 tempest-VolumeQuotaClassesTest-918343840-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:07.092044 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b4e1e0-d061-447d-89eb-83aa568c1a3f tempest-VolumeQuotaClassesTest-918343840 tempest-VolumeQuotaClassesTest-918343840-project-admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:07.104872 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 947/1879] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 01:57:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:57:07.151333 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793d63ed-8418-4f65-bebe-96c71249cb99 tempest-VolumeQuotaClassesTest-918343840 tempest-VolumeQuotaClassesTest-918343840-project-admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:07.162561 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793d63ed-8418-4f65-bebe-96c71249cb99 tempest-VolumeQuotaClassesTest-918343840 tempest-VolumeQuotaClassesTest-918343840-project-admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:07.168360 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b04ec-7168-4e94-be22-fd35e94398df tempest-VolumeQuotaClassesTest-918343840 tempest-VolumeQuotaClassesTest-918343840-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac63f3f2a81945e7b7af62848d179da4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0592e3e6647415f98b273e1d8833887', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:07.169232 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b04ec-7168-4e94-be22-fd35e94398df tempest-VolumeQuotaClassesTest-918343840 tempest-VolumeQuotaClassesTest-918343840-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:07.169513 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b04ec-7168-4e94-be22-fd35e94398df tempest-VolumeQuotaClassesTest-918343840 tempest-VolumeQuotaClassesTest-918343840-project-admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:07.169788 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b04ec-7168-4e94-be22-fd35e94398df tempest-VolumeQuotaClassesTest-918343840 tempest-VolumeQuotaClassesTest-918343840-project-admin] PATH_INFO: `/v3/projects/6603e223805e4a52bb395abc1833b458` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:07.171017 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132b04ec-7168-4e94-be22-fd35e94398df tempest-VolumeQuotaClassesTest-918343840 tempest-VolumeQuotaClassesTest-918343840-project-admin] RBAC: Authorizing `identity:delete_project(project_id=6603e223805e4a52bb395abc1833b458)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:07.173108 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132b04ec-7168-4e94-be22-fd35e94398df tempest-VolumeQuotaClassesTest-918343840 tempest-VolumeQuotaClassesTest-918343840-project-admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:07.208224 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:07.224403 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-132b04ec-7168-4e94-be22-fd35e94398df tempest-VolumeQuotaClassesTest-918343840 tempest-VolumeQuotaClassesTest-918343840-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6603e223805e4a52bb395abc1833b458', 'request_id': 'req-132b04ec-7168-4e94-be22-fd35e94398df'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:07.230876 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-132b04ec-7168-4e94-be22-fd35e94398df tempest-VolumeQuotaClassesTest-918343840 tempest-VolumeQuotaClassesTest-918343840-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6603e223805e4a52bb395abc1833b458', 'request_id': 'req-132b04ec-7168-4e94-be22-fd35e94398df'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:07.238433 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-132b04ec-7168-4e94-be22-fd35e94398df tempest-VolumeQuotaClassesTest-918343840 tempest-VolumeQuotaClassesTest-918343840-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6603e223805e4a52bb395abc1833b458', 'request_id': 'req-132b04ec-7168-4e94-be22-fd35e94398df'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:07.248651 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-132b04ec-7168-4e94-be22-fd35e94398df tempest-VolumeQuotaClassesTest-918343840 tempest-VolumeQuotaClassesTest-918343840-project-admin] The token cache is being invalidate because project 6603e223805e4a52bb395abc1833b458 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:57:07.248936 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-132b04ec-7168-4e94-be22-fd35e94398df tempest-VolumeQuotaClassesTest-918343840 tempest-VolumeQuotaClassesTest-918343840-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:07.249575 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-132b04ec-7168-4e94-be22-fd35e94398df tempest-VolumeQuotaClassesTest-918343840 tempest-VolumeQuotaClassesTest-918343840-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:07.251158 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 933/1880] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:57:07 2022] DELETE /identity/v3/projects/6603e223805e4a52bb395abc1833b458 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:07.448609 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b4e1e0-d061-447d-89eb-83aa568c1a3f tempest-VolumeQuotaClassesTest-918343840 tempest-VolumeQuotaClassesTest-918343840-project-admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:07.450365 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-70b4e1e0-d061-447d-89eb-83aa568c1a3f tempest-VolumeQuotaClassesTest-918343840 tempest-VolumeQuotaClassesTest-918343840-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:07.515530 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b4e1e0-d061-447d-89eb-83aa568c1a3f tempest-VolumeQuotaClassesTest-918343840 tempest-VolumeQuotaClassesTest-918343840-project-admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:07.576079 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f7902-bb40-44cf-9df0-443d0832fd54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:07.576973 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f7902-bb40-44cf-9df0-443d0832fd54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:07.577488 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f7902-bb40-44cf-9df0-443d0832fd54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:07.577769 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f7902-bb40-44cf-9df0-443d0832fd54 admin admin] PATH_INFO: `/v3/users/7b094d1fb1324b65b581386f7af619b7` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:07.579567 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37f7902-bb40-44cf-9df0-443d0832fd54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b094d1fb1324b65b581386f7af619b7)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:07.581796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37f7902-bb40-44cf-9df0-443d0832fd54 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:07.640404 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-a37f7902-bb40-44cf-9df0-443d0832fd54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b094d1fb1324b65b581386f7af619b7'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:07.650740 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:07.661772 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-a37f7902-bb40-44cf-9df0-443d0832fd54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b094d1fb1324b65b581386f7af619b7'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:07.671974 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-a37f7902-bb40-44cf-9df0-443d0832fd54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b094d1fb1324b65b581386f7af619b7'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:07.683726 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-a37f7902-bb40-44cf-9df0-443d0832fd54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b094d1fb1324b65b581386f7af619b7'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:07.684825 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-a37f7902-bb40-44cf-9df0-443d0832fd54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b094d1fb1324b65b581386f7af619b7'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:07.697092 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 948/1881] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:57:07 2022] DELETE /identity/v3/users/7b094d1fb1324b65b581386f7af619b7 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:07.706056 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e35979-d1bc-4232-a5c2-2953bc973728 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:07.706582 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e35979-d1bc-4232-a5c2-2953bc973728 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:07.706582 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e35979-d1bc-4232-a5c2-2953bc973728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:07.727546 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-d6e35979-d1bc-4232-a5c2-2953bc973728 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:07.729092 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-d6e35979-d1bc-4232-a5c2-2953bc973728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:07.822315 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 934/1882] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 01:57:07.840161 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f7902-bb40-44cf-9df0-443d0832fd54 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:07.842239 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-a37f7902-bb40-44cf-9df0-443d0832fd54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:07.922603 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f7902-bb40-44cf-9df0-443d0832fd54 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:07.926623 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae98423f-e41a-4590-8109-d6e1f2ef1a43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:07.927932 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98423f-e41a-4590-8109-d6e1f2ef1a43 service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:07.937465 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98423f-e41a-4590-8109-d6e1f2ef1a43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:07.937843 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98423f-e41a-4590-8109-d6e1f2ef1a43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:07.960806 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae98423f-e41a-4590-8109-d6e1f2ef1a43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:07.962923 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae98423f-e41a-4590-8109-d6e1f2ef1a43 service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:08.002196 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 949/1883] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:57:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:57:08.447970 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e35979-d1bc-4232-a5c2-2953bc973728 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:08.451061 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-d6e35979-d1bc-4232-a5c2-2953bc973728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:08.464388 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e35979-d1bc-4232-a5c2-2953bc973728 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:08.469514 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5976193-f7bd-46d1-9c6a-53a609fa376b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:08.470401 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5976193-f7bd-46d1-9c6a-53a609fa376b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:08.470859 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5976193-f7bd-46d1-9c6a-53a609fa376b admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:08.471188 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5976193-f7bd-46d1-9c6a-53a609fa376b admin admin] PATH_INFO: `/v3/projects/21ee263f4d7e4f7c94d4f521e78d8480` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:08.472464 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5976193-f7bd-46d1-9c6a-53a609fa376b admin admin] RBAC: Authorizing `identity:delete_project(project_id=21ee263f4d7e4f7c94d4f521e78d8480)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:08.474518 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5976193-f7bd-46d1-9c6a-53a609fa376b admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:08.545001 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-f5976193-f7bd-46d1-9c6a-53a609fa376b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21ee263f4d7e4f7c94d4f521e78d8480', 'request_id': 'req-f5976193-f7bd-46d1-9c6a-53a609fa376b'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:08.553225 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-f5976193-f7bd-46d1-9c6a-53a609fa376b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21ee263f4d7e4f7c94d4f521e78d8480', 'request_id': 'req-f5976193-f7bd-46d1-9c6a-53a609fa376b'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:08.561271 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-f5976193-f7bd-46d1-9c6a-53a609fa376b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21ee263f4d7e4f7c94d4f521e78d8480', 'request_id': 'req-f5976193-f7bd-46d1-9c6a-53a609fa376b'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:08.575498 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-f5976193-f7bd-46d1-9c6a-53a609fa376b admin admin] The token cache is being invalidate because project 21ee263f4d7e4f7c94d4f521e78d8480 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:57:08.575782 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-f5976193-f7bd-46d1-9c6a-53a609fa376b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:08.576517 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-f5976193-f7bd-46d1-9c6a-53a609fa376b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:08.580871 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 935/1884] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:57:08 2022] DELETE /identity/v3/projects/21ee263f4d7e4f7c94d4f521e78d8480 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:08.585616 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae98423f-e41a-4590-8109-d6e1f2ef1a43 service neutron] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:08.597678 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae98423f-e41a-4590-8109-d6e1f2ef1a43 service neutron] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:08.659714 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a5ce9c-90fe-4c22-afb4-227d7171e94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:08.660657 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5ce9c-90fe-4c22-afb4-227d7171e94a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:08.660939 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5ce9c-90fe-4c22-afb4-227d7171e94a admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:08.661283 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5ce9c-90fe-4c22-afb4-227d7171e94a admin admin] PATH_INFO: `/v3/users/ac63f3f2a81945e7b7af62848d179da4` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:08.662547 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a5ce9c-90fe-4c22-afb4-227d7171e94a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac63f3f2a81945e7b7af62848d179da4)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:08.664628 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a5ce9c-90fe-4c22-afb4-227d7171e94a admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:08.718313 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-22a5ce9c-90fe-4c22-afb4-227d7171e94a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac63f3f2a81945e7b7af62848d179da4'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:08.732952 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-22a5ce9c-90fe-4c22-afb4-227d7171e94a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac63f3f2a81945e7b7af62848d179da4'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:08.741060 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-22a5ce9c-90fe-4c22-afb4-227d7171e94a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac63f3f2a81945e7b7af62848d179da4'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:08.746661 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:08.749598 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-22a5ce9c-90fe-4c22-afb4-227d7171e94a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac63f3f2a81945e7b7af62848d179da4'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:08.750841 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-22a5ce9c-90fe-4c22-afb4-227d7171e94a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac63f3f2a81945e7b7af62848d179da4'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:08.758964 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 950/1885] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:57:08 2022] DELETE /identity/v3/users/ac63f3f2a81945e7b7af62848d179da4 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:09.267547 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5976193-f7bd-46d1-9c6a-53a609fa376b admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:09.270960 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-f5976193-f7bd-46d1-9c6a-53a609fa376b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:09.301035 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5976193-f7bd-46d1-9c6a-53a609fa376b admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:09.367506 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:09.425349 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d12948-0de5-4674-9f98-da9dbbd4b946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:09.426323 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d12948-0de5-4674-9f98-da9dbbd4b946 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:09.426655 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d12948-0de5-4674-9f98-da9dbbd4b946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:09.427015 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d12948-0de5-4674-9f98-da9dbbd4b946 admin admin] PATH_INFO: `/v3/projects/f0592e3e6647415f98b273e1d8833887` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:09.429727 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d12948-0de5-4674-9f98-da9dbbd4b946 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0592e3e6647415f98b273e1d8833887)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:09.432441 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d12948-0de5-4674-9f98-da9dbbd4b946 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:09.499246 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-29d12948-0de5-4674-9f98-da9dbbd4b946 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0592e3e6647415f98b273e1d8833887', 'request_id': 'req-29d12948-0de5-4674-9f98-da9dbbd4b946'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:09.506449 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-29d12948-0de5-4674-9f98-da9dbbd4b946 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0592e3e6647415f98b273e1d8833887', 'request_id': 'req-29d12948-0de5-4674-9f98-da9dbbd4b946'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:09.518270 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-29d12948-0de5-4674-9f98-da9dbbd4b946 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0592e3e6647415f98b273e1d8833887', 'request_id': 'req-29d12948-0de5-4674-9f98-da9dbbd4b946'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:09.538570 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-29d12948-0de5-4674-9f98-da9dbbd4b946 admin admin] The token cache is being invalidate because project f0592e3e6647415f98b273e1d8833887 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:57:09.539033 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-29d12948-0de5-4674-9f98-da9dbbd4b946 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:09.540661 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-29d12948-0de5-4674-9f98-da9dbbd4b946 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:09.542552 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 936/1886] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:57:09 2022] DELETE /identity/v3/projects/f0592e3e6647415f98b273e1d8833887 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:09.549640 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f72847-34b9-4205-a360-35e5c080cf2b None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:09.550548 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f72847-34b9-4205-a360-35e5c080cf2b None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:09.551006 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f72847-34b9-4205-a360-35e5c080cf2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:09.588605 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-80f72847-34b9-4205-a360-35e5c080cf2b None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:09.589943 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-80f72847-34b9-4205-a360-35e5c080cf2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:09.694122 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 951/1887] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:09.701579 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202baec5-5ce4-4cce-bf3c-97dccdd185cb None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:09.701870 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202baec5-5ce4-4cce-bf3c-97dccdd185cb None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:09.702151 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202baec5-5ce4-4cce-bf3c-97dccdd185cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:09.736869 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-202baec5-5ce4-4cce-bf3c-97dccdd185cb None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:09.739719 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-202baec5-5ce4-4cce-bf3c-97dccdd185cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:09.758462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 937/1888] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:09.760048 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f72847-34b9-4205-a360-35e5c080cf2b None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:09.775393 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f72847-34b9-4205-a360-35e5c080cf2b None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:09.793276 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc5842-89b0-4e05-b2f6-a1c9c123fdbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:09.794027 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc5842-89b0-4e05-b2f6-a1c9c123fdbb admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:09.794311 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc5842-89b0-4e05-b2f6-a1c9c123fdbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:09.794589 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc5842-89b0-4e05-b2f6-a1c9c123fdbb admin admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:09.795913 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfc5842-89b0-4e05-b2f6-a1c9c123fdbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:57:09.796225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfc5842-89b0-4e05-b2f6-a1c9c123fdbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:09.797870 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfc5842-89b0-4e05-b2f6-a1c9c123fdbb admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:09.832671 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 952/1889] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 01:57:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:57:09.834565 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202baec5-5ce4-4cce-bf3c-97dccdd185cb None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:09.856267 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202baec5-5ce4-4cce-bf3c-97dccdd185cb None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:09.863063 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbdffcd-7d62-484c-b735-e7dce1c5429d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:09.863850 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbdffcd-7d62-484c-b735-e7dce1c5429d admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:09.864060 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbdffcd-7d62-484c-b735-e7dce1c5429d admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:09.864412 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbdffcd-7d62-484c-b735-e7dce1c5429d admin admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:09.865141 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbdffcd-7d62-484c-b735-e7dce1c5429d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:09.867289 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbdffcd-7d62-484c-b735-e7dce1c5429d admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:09.907743 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 938/1890] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:57:09 2022] POST /identity/v3/projects => generated 393 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:57:09.908881 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc5842-89b0-4e05-b2f6-a1c9c123fdbb admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:09.927661 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc5842-89b0-4e05-b2f6-a1c9c123fdbb admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:10.041699 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ebcf2f-b975-4405-8fc8-dd12a8735a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:10.043923 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ebcf2f-b975-4405-8fc8-dd12a8735a7e admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:10.044483 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ebcf2f-b975-4405-8fc8-dd12a8735a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:10.044961 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ebcf2f-b975-4405-8fc8-dd12a8735a7e admin admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:10.046417 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ebcf2f-b975-4405-8fc8-dd12a8735a7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:10.049452 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ebcf2f-b975-4405-8fc8-dd12a8735a7e admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:10.091628 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 953/1891] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:57:09 2022] POST /identity/v3/users => generated 297 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:57:10.095570 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbdffcd-7d62-484c-b735-e7dce1c5429d admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:10.119831 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbdffcd-7d62-484c-b735-e7dce1c5429d admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:10.136042 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc59cde4-67a2-407c-bcd6-7c8b2f422847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:10.138130 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc59cde4-67a2-407c-bcd6-7c8b2f422847 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:10.138562 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc59cde4-67a2-407c-bcd6-7c8b2f422847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:10.139077 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc59cde4-67a2-407c-bcd6-7c8b2f422847 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:10.140408 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc59cde4-67a2-407c-bcd6-7c8b2f422847 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:10.143181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc59cde4-67a2-407c-bcd6-7c8b2f422847 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:10.188463 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 939/1892] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:57:10 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:57:10.193611 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ebcf2f-b975-4405-8fc8-dd12a8735a7e admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:10.216139 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ebcf2f-b975-4405-8fc8-dd12a8735a7e admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:10.234326 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc35869-bfff-404d-bc45-41b2fd54a2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:10.235071 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc35869-bfff-404d-bc45-41b2fd54a2ef admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:10.235383 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc35869-bfff-404d-bc45-41b2fd54a2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:10.235705 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc35869-bfff-404d-bc45-41b2fd54a2ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:10.236464 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc35869-bfff-404d-bc45-41b2fd54a2ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:10.238190 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc35869-bfff-404d-bc45-41b2fd54a2ef admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:10.256972 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 954/1893] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:57:10 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 01:57:10.265233 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc59cde4-67a2-407c-bcd6-7c8b2f422847 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:10.277339 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc59cde4-67a2-407c-bcd6-7c8b2f422847 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:10.296560 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8eaf86-f03a-4a2b-be79-fc023afd3cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:10.300151 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8eaf86-f03a-4a2b-be79-fc023afd3cce admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:10.300731 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8eaf86-f03a-4a2b-be79-fc023afd3cce admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:10.301005 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8eaf86-f03a-4a2b-be79-fc023afd3cce admin admin] PATH_INFO: `/v3/projects/ba26dcd960594370b237a9adf3b8f7d6/users/4a01a8cfeaac46c79a8a542c45d865fe/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:10.357717 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8eaf86-f03a-4a2b-be79-fc023afd3cce admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba26dcd960594370b237a9adf3b8f7d6, user_id=4a01a8cfeaac46c79a8a542c45d865fe, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:10.364306 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8eaf86-f03a-4a2b-be79-fc023afd3cce admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:10.391300 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 940/1894] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:57:10 2022] PUT /identity/v3/projects/ba26dcd960594370b237a9adf3b8f7d6/users/4a01a8cfeaac46c79a8a542c45d865fe/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:10.402520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5379db6-8515-47cf-a47d-b012f572ac18 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:10.402873 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5379db6-8515-47cf-a47d-b012f572ac18 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:10.403177 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5379db6-8515-47cf-a47d-b012f572ac18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:10.443720 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-a5379db6-8515-47cf-a47d-b012f572ac18 None None] MFA Rules not processed for user `4a01a8cfeaac46c79a8a542c45d865fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:10.445057 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-a5379db6-8515-47cf-a47d-b012f572ac18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:10.534727 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:10.607680 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8eaf86-f03a-4a2b-be79-fc023afd3cce admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:10.608278 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 955/1895] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:10.736676 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8eaf86-f03a-4a2b-be79-fc023afd3cce admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:10.742770 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c80b58-8dd0-42db-b44f-a07aa1fb2186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:10.743594 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c80b58-8dd0-42db-b44f-a07aa1fb2186 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:10.743812 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c80b58-8dd0-42db-b44f-a07aa1fb2186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:10.744041 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c80b58-8dd0-42db-b44f-a07aa1fb2186 admin admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:10.745528 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c80b58-8dd0-42db-b44f-a07aa1fb2186 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:10.747916 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c80b58-8dd0-42db-b44f-a07aa1fb2186 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:10.761986 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:10.786308 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 941/1896] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:57:10 2022] POST /identity/v3/projects => generated 393 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 01:57:10.789946 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5379db6-8515-47cf-a47d-b012f572ac18 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:10.808720 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5379db6-8515-47cf-a47d-b012f572ac18 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:10.886795 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0e59ae-2e51-44d4-b5f8-6d823e1c4a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:10.887989 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0e59ae-2e51-44d4-b5f8-6d823e1c4a6a admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:10.887989 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0e59ae-2e51-44d4-b5f8-6d823e1c4a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:10.888243 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0e59ae-2e51-44d4-b5f8-6d823e1c4a6a admin admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:10.888997 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0e59ae-2e51-44d4-b5f8-6d823e1c4a6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:10.891348 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0e59ae-2e51-44d4-b5f8-6d823e1c4a6a admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:10.919553 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 956/1897] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:57:10 2022] POST /identity/v3/users => generated 303 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:57:10.923403 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c80b58-8dd0-42db-b44f-a07aa1fb2186 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:10.940744 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c80b58-8dd0-42db-b44f-a07aa1fb2186 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:10.949451 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42155a6-fba3-4ae7-b704-ae049de4c95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:10.950227 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42155a6-fba3-4ae7-b704-ae049de4c95d admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:10.950518 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42155a6-fba3-4ae7-b704-ae049de4c95d admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:10.950868 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42155a6-fba3-4ae7-b704-ae049de4c95d admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:10.953073 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42155a6-fba3-4ae7-b704-ae049de4c95d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:10.955526 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42155a6-fba3-4ae7-b704-ae049de4c95d admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:10.971065 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 942/1898] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:57:10 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:57:10.972425 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0e59ae-2e51-44d4-b5f8-6d823e1c4a6a admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:10.982466 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0e59ae-2e51-44d4-b5f8-6d823e1c4a6a admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:10.987018 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b73e6-eff0-41ab-bbfa-e7f9e5b1cf40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:10.987954 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b73e6-eff0-41ab-bbfa-e7f9e5b1cf40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:10.988233 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b73e6-eff0-41ab-bbfa-e7f9e5b1cf40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:10.988507 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b73e6-eff0-41ab-bbfa-e7f9e5b1cf40 admin admin] PATH_INFO: `/v3/projects/3044ee619657457f8fd7734fdf86e7bf/users/a31eb86c055a4e959286f37c594a0019/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:11.009608 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7b73e6-eff0-41ab-bbfa-e7f9e5b1cf40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3044ee619657457f8fd7734fdf86e7bf, user_id=a31eb86c055a4e959286f37c594a0019, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:11.015890 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7b73e6-eff0-41ab-bbfa-e7f9e5b1cf40 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:11.042763 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 957/1899] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:57:10 2022] PUT /identity/v3/projects/3044ee619657457f8fd7734fdf86e7bf/users/a31eb86c055a4e959286f37c594a0019/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:11.044117 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42155a6-fba3-4ae7-b704-ae049de4c95d admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:11.061566 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42155a6-fba3-4ae7-b704-ae049de4c95d admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:11.131006 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f0b456-3799-44bc-8bbe-7c1a35d043fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:11.131835 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0b456-3799-44bc-8bbe-7c1a35d043fe admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:11.132098 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0b456-3799-44bc-8bbe-7c1a35d043fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:11.132366 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0b456-3799-44bc-8bbe-7c1a35d043fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:11.133102 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f0b456-3799-44bc-8bbe-7c1a35d043fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:11.134695 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f0b456-3799-44bc-8bbe-7c1a35d043fe admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:11.147126 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 943/1900] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:57:11 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:57:11.151518 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b73e6-eff0-41ab-bbfa-e7f9e5b1cf40 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:11.162876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b73e6-eff0-41ab-bbfa-e7f9e5b1cf40 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:11.168550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51769a75-1a29-4079-881b-ae65fd1f24d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:11.169537 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51769a75-1a29-4079-881b-ae65fd1f24d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:11.169888 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51769a75-1a29-4079-881b-ae65fd1f24d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:11.170241 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51769a75-1a29-4079-881b-ae65fd1f24d7 admin admin] PATH_INFO: `/v3/projects/3044ee619657457f8fd7734fdf86e7bf/users/a31eb86c055a4e959286f37c594a0019/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:11.172364 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51769a75-1a29-4079-881b-ae65fd1f24d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3044ee619657457f8fd7734fdf86e7bf, user_id=a31eb86c055a4e959286f37c594a0019, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:11.175522 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51769a75-1a29-4079-881b-ae65fd1f24d7 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:11.200998 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 958/1901] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:57:11 2022] PUT /identity/v3/projects/3044ee619657457f8fd7734fdf86e7bf/users/a31eb86c055a4e959286f37c594a0019/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:11.210773 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51769a75-1a29-4079-881b-ae65fd1f24d7 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:11.211359 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437324f-6926-40db-ab23-b68837d7cd6a None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:11.211643 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437324f-6926-40db-ab23-b68837d7cd6a None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:11.211925 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437324f-6926-40db-ab23-b68837d7cd6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:11.212990 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-51769a75-1a29-4079-881b-ae65fd1f24d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:11.240562 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-0437324f-6926-40db-ab23-b68837d7cd6a None None] MFA Rules not processed for user `a31eb86c055a4e959286f37c594a0019`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:11.241962 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-0437324f-6926-40db-ab23-b68837d7cd6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:11.321921 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51769a75-1a29-4079-881b-ae65fd1f24d7 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:11.327180 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a1f54a-422f-4900-b4ff-d2da3dc7eb54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:11.328226 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1f54a-422f-4900-b4ff-d2da3dc7eb54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:11.328514 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1f54a-422f-4900-b4ff-d2da3dc7eb54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:11.328790 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1f54a-422f-4900-b4ff-d2da3dc7eb54 admin admin] PATH_INFO: `/v3/users/5d2a25e6872641b6824b00633b7d2117` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:11.330648 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a1f54a-422f-4900-b4ff-d2da3dc7eb54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d2a25e6872641b6824b00633b7d2117)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:11.332681 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a1f54a-422f-4900-b4ff-d2da3dc7eb54 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:11.345161 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 944/1902] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:11 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:11.357650 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0437324f-6926-40db-ab23-b68837d7cd6a None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:11.359060 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-0437324f-6926-40db-ab23-b68837d7cd6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:11.418960 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:11.418960 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-25a1f54a-422f-4900-b4ff-d2da3dc7eb54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d2a25e6872641b6824b00633b7d2117'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:11.463765 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0437324f-6926-40db-ab23-b68837d7cd6a None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:11.464014 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-25a1f54a-422f-4900-b4ff-d2da3dc7eb54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d2a25e6872641b6824b00633b7d2117'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:11.468202 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dd9cc1-b222-4b97-9c42-f7b91782d0ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:11.469271 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd9cc1-b222-4b97-9c42-f7b91782d0ba service cinder] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:11.469679 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd9cc1-b222-4b97-9c42-f7b91782d0ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:11.470085 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd9cc1-b222-4b97-9c42-f7b91782d0ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:11.477166 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-25a1f54a-422f-4900-b4ff-d2da3dc7eb54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d2a25e6872641b6824b00633b7d2117'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:11.484760 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-25a1f54a-422f-4900-b4ff-d2da3dc7eb54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d2a25e6872641b6824b00633b7d2117'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:11.485455 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-25a1f54a-422f-4900-b4ff-d2da3dc7eb54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d2a25e6872641b6824b00633b7d2117'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:11.491973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dd9cc1-b222-4b97-9c42-f7b91782d0ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:11.494440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dd9cc1-b222-4b97-9c42-f7b91782d0ba service cinder] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:11.496867 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 959/1903] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:57:11 2022] DELETE /identity/v3/users/5d2a25e6872641b6824b00633b7d2117 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:11.502173 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c24fa-7a67-4576-8c45-a1406a9f3ec7 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:11.503524 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c24fa-7a67-4576-8c45-a1406a9f3ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:11.505290 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c24fa-7a67-4576-8c45-a1406a9f3ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:11.524341 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 945/1904] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:57:11 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:57:11.558957 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-256c24fa-7a67-4576-8c45-a1406a9f3ec7 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:11.561113 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-256c24fa-7a67-4576-8c45-a1406a9f3ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:11.629915 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dd9cc1-b222-4b97-9c42-f7b91782d0ba service cinder] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:11.638525 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:11.646276 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dd9cc1-b222-4b97-9c42-f7b91782d0ba service cinder] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:11.688550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 960/1905] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:11.709001 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256c24fa-7a67-4576-8c45-a1406a9f3ec7 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:11.710421 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-256c24fa-7a67-4576-8c45-a1406a9f3ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:11.780856 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75d91e-f05a-488a-841c-fe9f00c43c02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:11.783372 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75d91e-f05a-488a-841c-fe9f00c43c02 service cinder] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:11.784767 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75d91e-f05a-488a-841c-fe9f00c43c02 service cinder] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:11.785462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75d91e-f05a-488a-841c-fe9f00c43c02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:11.789774 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-9c75d91e-f05a-488a-841c-fe9f00c43c02 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:11.822647 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256c24fa-7a67-4576-8c45-a1406a9f3ec7 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:11.826946 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dc9265-e85a-41bb-a59a-8b7d9fd2875b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:11.827762 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc9265-e85a-41bb-a59a-8b7d9fd2875b service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:11.828324 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc9265-e85a-41bb-a59a-8b7d9fd2875b service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:11.828718 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc9265-e85a-41bb-a59a-8b7d9fd2875b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:11.843468 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dc9265-e85a-41bb-a59a-8b7d9fd2875b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:11.845949 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dc9265-e85a-41bb-a59a-8b7d9fd2875b service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:11.868988 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 961/1906] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:57:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 01:57:11.908413 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c75d91e-f05a-488a-841c-fe9f00c43c02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:11.910904 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c75d91e-f05a-488a-841c-fe9f00c43c02 service cinder] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:11.928047 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 946/1907] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:57:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 01:57:12.184918 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882928d-fc2f-4d8f-908f-881a2d14327f None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:12.185244 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882928d-fc2f-4d8f-908f-881a2d14327f None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:12.185508 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882928d-fc2f-4d8f-908f-881a2d14327f None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:12.186977 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 962/1908] 10.222.0.26 () {54 vars in 921 bytes} [Thu Jun 23 01:57:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:57:12.200035 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6fabf-2e43-4887-8331-7e9af2bf8bc4 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:12.200442 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6fabf-2e43-4887-8331-7e9af2bf8bc4 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:12.200775 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6fabf-2e43-4887-8331-7e9af2bf8bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:12.268194 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-78f6fabf-2e43-4887-8331-7e9af2bf8bc4 None None] MFA Rules not processed for user `ae4e83df33b64a3ebf28f54e2d94fe63`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:12.272456 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-78f6fabf-2e43-4887-8331-7e9af2bf8bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:12.381139 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 947/1909] 10.222.0.26 () {58 vars in 1018 bytes} [Thu Jun 23 01:57:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:12.400515 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b882928d-fc2f-4d8f-908f-881a2d14327f None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:12.402708 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-b882928d-fc2f-4d8f-908f-881a2d14327f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:12.428323 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b882928d-fc2f-4d8f-908f-881a2d14327f None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:12.450625 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59829837-cb35-4dfe-acd5-2bdbeb73558c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4e83df33b64a3ebf28f54e2d94fe63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:12.453104 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59829837-cb35-4dfe-acd5-2bdbeb73558c service nova] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:12.453423 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59829837-cb35-4dfe-acd5-2bdbeb73558c service nova] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:12.453864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59829837-cb35-4dfe-acd5-2bdbeb73558c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:12.482559 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59829837-cb35-4dfe-acd5-2bdbeb73558c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:12.485337 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59829837-cb35-4dfe-acd5-2bdbeb73558c service nova] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:12.520917 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 963/1910] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:57:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:57:13.078834 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f6fabf-2e43-4887-8331-7e9af2bf8bc4 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:13.080295 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-78f6fabf-2e43-4887-8331-7e9af2bf8bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:13.097530 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f6fabf-2e43-4887-8331-7e9af2bf8bc4 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:13.105084 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c13ce6-dcd7-4e35-a21f-626c827a3373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:13.106118 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c13ce6-dcd7-4e35-a21f-626c827a3373 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:13.106503 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c13ce6-dcd7-4e35-a21f-626c827a3373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:13.106732 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c13ce6-dcd7-4e35-a21f-626c827a3373 admin admin] PATH_INFO: `/v3/projects/3142b780b5354cd0ac4e3a88d4233562` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:13.112894 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c13ce6-dcd7-4e35-a21f-626c827a3373 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3142b780b5354cd0ac4e3a88d4233562)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:13.114943 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c13ce6-dcd7-4e35-a21f-626c827a3373 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:13.223291 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:13.227508 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-83c13ce6-dcd7-4e35-a21f-626c827a3373 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3142b780b5354cd0ac4e3a88d4233562', 'request_id': 'req-83c13ce6-dcd7-4e35-a21f-626c827a3373'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:13.237119 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-83c13ce6-dcd7-4e35-a21f-626c827a3373 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3142b780b5354cd0ac4e3a88d4233562', 'request_id': 'req-83c13ce6-dcd7-4e35-a21f-626c827a3373'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:13.248440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-83c13ce6-dcd7-4e35-a21f-626c827a3373 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3142b780b5354cd0ac4e3a88d4233562', 'request_id': 'req-83c13ce6-dcd7-4e35-a21f-626c827a3373'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:13.273398 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-83c13ce6-dcd7-4e35-a21f-626c827a3373 admin admin] The token cache is being invalidate because project 3142b780b5354cd0ac4e3a88d4233562 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:57:13.273820 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-83c13ce6-dcd7-4e35-a21f-626c827a3373 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:13.274361 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-83c13ce6-dcd7-4e35-a21f-626c827a3373 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:13.276580 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 948/1911] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:57:13 2022] DELETE /identity/v3/projects/3142b780b5354cd0ac4e3a88d4233562 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:13.281224 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59829837-cb35-4dfe-acd5-2bdbeb73558c service nova] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:13.301144 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59829837-cb35-4dfe-acd5-2bdbeb73558c service nova] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:13.416203 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:13.433183 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc18910d-a18e-4c96-b8b2-7006ea8216a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:13.434360 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc18910d-a18e-4c96-b8b2-7006ea8216a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:13.434792 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc18910d-a18e-4c96-b8b2-7006ea8216a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:13.435619 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc18910d-a18e-4c96-b8b2-7006ea8216a6 admin admin] PATH_INFO: `/v3/users/4d8b027d22a1435ea591392f5bee8453` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:13.437966 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc18910d-a18e-4c96-b8b2-7006ea8216a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d8b027d22a1435ea591392f5bee8453)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:13.440603 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc18910d-a18e-4c96-b8b2-7006ea8216a6 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:13.501850 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-cc18910d-a18e-4c96-b8b2-7006ea8216a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d8b027d22a1435ea591392f5bee8453'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:13.524103 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-cc18910d-a18e-4c96-b8b2-7006ea8216a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d8b027d22a1435ea591392f5bee8453'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:13.532544 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-cc18910d-a18e-4c96-b8b2-7006ea8216a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d8b027d22a1435ea591392f5bee8453'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:13.539573 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-cc18910d-a18e-4c96-b8b2-7006ea8216a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d8b027d22a1435ea591392f5bee8453'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:13.540191 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-cc18910d-a18e-4c96-b8b2-7006ea8216a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d8b027d22a1435ea591392f5bee8453'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:13.546053 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 964/1912] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:57:13 2022] DELETE /identity/v3/users/4d8b027d22a1435ea591392f5bee8453 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:13.970539 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c13ce6-dcd7-4e35-a21f-626c827a3373 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:13.972212 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-83c13ce6-dcd7-4e35-a21f-626c827a3373 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:14.005776 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c13ce6-dcd7-4e35-a21f-626c827a3373 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:14.128219 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab0d74-a0d8-4ca9-8843-45319f2d0da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:14.129138 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab0d74-a0d8-4ca9-8843-45319f2d0da5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:14.129406 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab0d74-a0d8-4ca9-8843-45319f2d0da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:14.129679 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab0d74-a0d8-4ca9-8843-45319f2d0da5 admin admin] PATH_INFO: `/v3/projects/1fa87b8ef18a411b83618c37319756c3` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:14.137177 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ab0d74-a0d8-4ca9-8843-45319f2d0da5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fa87b8ef18a411b83618c37319756c3)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:14.138073 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ab0d74-a0d8-4ca9-8843-45319f2d0da5 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:14.242469 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-a6ab0d74-a0d8-4ca9-8843-45319f2d0da5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fa87b8ef18a411b83618c37319756c3', 'request_id': 'req-a6ab0d74-a0d8-4ca9-8843-45319f2d0da5'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:14.254070 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-a6ab0d74-a0d8-4ca9-8843-45319f2d0da5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fa87b8ef18a411b83618c37319756c3', 'request_id': 'req-a6ab0d74-a0d8-4ca9-8843-45319f2d0da5'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:14.279024 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-a6ab0d74-a0d8-4ca9-8843-45319f2d0da5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fa87b8ef18a411b83618c37319756c3', 'request_id': 'req-a6ab0d74-a0d8-4ca9-8843-45319f2d0da5'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:14.321773 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-a6ab0d74-a0d8-4ca9-8843-45319f2d0da5 admin admin] The token cache is being invalidate because project 1fa87b8ef18a411b83618c37319756c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:57:14.322066 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-a6ab0d74-a0d8-4ca9-8843-45319f2d0da5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:14.327637 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-a6ab0d74-a0d8-4ca9-8843-45319f2d0da5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:14.331354 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 949/1913] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:57:13 2022] DELETE /identity/v3/projects/1fa87b8ef18a411b83618c37319756c3 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:14.342506 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0371d3-6272-4994-9537-a753a13cb1a8 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:14.343376 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0371d3-6272-4994-9537-a753a13cb1a8 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:14.344375 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0371d3-6272-4994-9537-a753a13cb1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:14.370529 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-af0371d3-6272-4994-9537-a753a13cb1a8 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:14.372593 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-af0371d3-6272-4994-9537-a753a13cb1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:14.489096 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 965/1914] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:14.499255 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a681a2e-8a22-4dfa-967a-4d68ebfb14d9 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:14.499523 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a681a2e-8a22-4dfa-967a-4d68ebfb14d9 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:14.499802 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a681a2e-8a22-4dfa-967a-4d68ebfb14d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:14.546153 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-7a681a2e-8a22-4dfa-967a-4d68ebfb14d9 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:14.547377 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-7a681a2e-8a22-4dfa-967a-4d68ebfb14d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:14.579959 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 950/1915] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:14.580804 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0371d3-6272-4994-9537-a753a13cb1a8 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:14.593089 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0371d3-6272-4994-9537-a753a13cb1a8 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:14.611613 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce84eca-0198-411e-a47d-fcbdca0f4ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:14.612294 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce84eca-0198-411e-a47d-fcbdca0f4ff2 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:14.612578 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce84eca-0198-411e-a47d-fcbdca0f4ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:14.612854 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce84eca-0198-411e-a47d-fcbdca0f4ff2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:14.621920 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce84eca-0198-411e-a47d-fcbdca0f4ff2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:57:14.622215 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce84eca-0198-411e-a47d-fcbdca0f4ff2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:14.631685 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce84eca-0198-411e-a47d-fcbdca0f4ff2 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:14.662532 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 966/1916] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 01:57:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:57:14.667403 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a681a2e-8a22-4dfa-967a-4d68ebfb14d9 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:14.679314 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a681a2e-8a22-4dfa-967a-4d68ebfb14d9 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:14.688807 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd97d7e-0d8c-4779-b890-ca2a3144a854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:14.689863 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd97d7e-0d8c-4779-b890-ca2a3144a854 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:14.690306 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd97d7e-0d8c-4779-b890-ca2a3144a854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:14.692657 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd97d7e-0d8c-4779-b890-ca2a3144a854 admin admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:14.695695 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd97d7e-0d8c-4779-b890-ca2a3144a854 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:14.700448 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd97d7e-0d8c-4779-b890-ca2a3144a854 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:14.775156 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 951/1917] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:57:14 2022] POST /identity/v3/projects => generated 367 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 01:57:14.779887 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce84eca-0198-411e-a47d-fcbdca0f4ff2 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:14.793911 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce84eca-0198-411e-a47d-fcbdca0f4ff2 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:14.878222 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:14.885916 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7706f0a9-651a-42ca-9da2-a141ffc20539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:14.886600 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706f0a9-651a-42ca-9da2-a141ffc20539 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:14.886873 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706f0a9-651a-42ca-9da2-a141ffc20539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:14.887230 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706f0a9-651a-42ca-9da2-a141ffc20539 admin admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:14.887995 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7706f0a9-651a-42ca-9da2-a141ffc20539 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:14.890465 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7706f0a9-651a-42ca-9da2-a141ffc20539 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:14.915390 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 967/1918] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:57:14 2022] POST /identity/v3/users => generated 284 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 01:57:14.919783 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd97d7e-0d8c-4779-b890-ca2a3144a854 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:14.930958 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd97d7e-0d8c-4779-b890-ca2a3144a854 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:14.935964 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010af5d5-4a09-4ce5-95ea-8cb924140bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:14.936765 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010af5d5-4a09-4ce5-95ea-8cb924140bd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:14.937107 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010af5d5-4a09-4ce5-95ea-8cb924140bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:14.937449 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010af5d5-4a09-4ce5-95ea-8cb924140bd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:14.938247 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010af5d5-4a09-4ce5-95ea-8cb924140bd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:14.940013 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010af5d5-4a09-4ce5-95ea-8cb924140bd1 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:14.951330 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 952/1919] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:57:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:57:14.956260 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7706f0a9-651a-42ca-9da2-a141ffc20539 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:14.966927 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7706f0a9-651a-42ca-9da2-a141ffc20539 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:14.972346 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a79ebf-e123-4eac-bf1b-7f816fa7f607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:14.973137 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a79ebf-e123-4eac-bf1b-7f816fa7f607 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:14.973458 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a79ebf-e123-4eac-bf1b-7f816fa7f607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:14.974100 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a79ebf-e123-4eac-bf1b-7f816fa7f607 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:14.974465 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a79ebf-e123-4eac-bf1b-7f816fa7f607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:14.976437 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a79ebf-e123-4eac-bf1b-7f816fa7f607 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:14.989825 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 968/1920] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:57:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 01:57:14.994106 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010af5d5-4a09-4ce5-95ea-8cb924140bd1 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:15.006482 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010af5d5-4a09-4ce5-95ea-8cb924140bd1 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:15.012210 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09333285-7e9e-4dee-bab1-12f0f2d3aeba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:15.013351 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09333285-7e9e-4dee-bab1-12f0f2d3aeba admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:15.013733 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09333285-7e9e-4dee-bab1-12f0f2d3aeba admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:15.014097 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09333285-7e9e-4dee-bab1-12f0f2d3aeba admin admin] PATH_INFO: `/v3/projects/7eb06b7c6795491899d9e0ca42564344/users/96f434251eef46958f16259266ac5b33/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:15.032215 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09333285-7e9e-4dee-bab1-12f0f2d3aeba admin admin] RBAC: Authorizing `identity:create_grant(project_id=7eb06b7c6795491899d9e0ca42564344, user_id=96f434251eef46958f16259266ac5b33, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:15.035518 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09333285-7e9e-4dee-bab1-12f0f2d3aeba admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:15.048775 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:15.055930 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 953/1921] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:57:14 2022] PUT /identity/v3/projects/7eb06b7c6795491899d9e0ca42564344/users/96f434251eef46958f16259266ac5b33/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:15.068444 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34311d4e-936b-436b-ada0-2d8155bff8a0 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:15.068725 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34311d4e-936b-436b-ada0-2d8155bff8a0 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:15.069028 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34311d4e-936b-436b-ada0-2d8155bff8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:15.103805 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-34311d4e-936b-436b-ada0-2d8155bff8a0 None None] MFA Rules not processed for user `96f434251eef46958f16259266ac5b33`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:15.105089 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-34311d4e-936b-436b-ada0-2d8155bff8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:15.198581 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 969/1922] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:15 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:15.207156 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09333285-7e9e-4dee-bab1-12f0f2d3aeba admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:15.209137 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-09333285-7e9e-4dee-bab1-12f0f2d3aeba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:15.268356 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09333285-7e9e-4dee-bab1-12f0f2d3aeba admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:15.273365 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49245e8-de46-4f4a-a111-aec43dbe72b0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c081b4cf34146ca8eb197b62a133433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:15.273608 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49245e8-de46-4f4a-a111-aec43dbe72b0 service glance] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:15.273892 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49245e8-de46-4f4a-a111-aec43dbe72b0 service glance] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:15.274163 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49245e8-de46-4f4a-a111-aec43dbe72b0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:15.282895 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49245e8-de46-4f4a-a111-aec43dbe72b0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:15.284344 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49245e8-de46-4f4a-a111-aec43dbe72b0 service glance] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:15.294469 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 954/1923] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:57:15 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:57:18.602067 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34311d4e-936b-436b-ada0-2d8155bff8a0 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:18.604239 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-34311d4e-936b-436b-ada0-2d8155bff8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:18.705293 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34311d4e-936b-436b-ada0-2d8155bff8a0 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:18.710705 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d66d3-65b3-4b67-980a-11c0519543fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:18.711718 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d66d3-65b3-4b67-980a-11c0519543fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:18.712065 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d66d3-65b3-4b67-980a-11c0519543fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:18.712425 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d66d3-65b3-4b67-980a-11c0519543fb admin admin] PATH_INFO: `/v3/users/4a01a8cfeaac46c79a8a542c45d865fe` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:18.713744 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24d66d3-65b3-4b67-980a-11c0519543fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a01a8cfeaac46c79a8a542c45d865fe)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:18.715794 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24d66d3-65b3-4b67-980a-11c0519543fb admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:18.765900 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-a24d66d3-65b3-4b67-980a-11c0519543fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a01a8cfeaac46c79a8a542c45d865fe'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:18.777601 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-a24d66d3-65b3-4b67-980a-11c0519543fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a01a8cfeaac46c79a8a542c45d865fe'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:18.780092 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:18.787449 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-a24d66d3-65b3-4b67-980a-11c0519543fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a01a8cfeaac46c79a8a542c45d865fe'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:18.797032 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-a24d66d3-65b3-4b67-980a-11c0519543fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a01a8cfeaac46c79a8a542c45d865fe'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:18.797747 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-a24d66d3-65b3-4b67-980a-11c0519543fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a01a8cfeaac46c79a8a542c45d865fe'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:18.805813 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 970/1924] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:57:18 2022] DELETE /identity/v3/users/4a01a8cfeaac46c79a8a542c45d865fe => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:18.810514 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6536b5c-07f8-4037-ad80-016588ed60d9 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:18.811087 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6536b5c-07f8-4037-ad80-016588ed60d9 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:18.811087 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6536b5c-07f8-4037-ad80-016588ed60d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:18.837350 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-e6536b5c-07f8-4037-ad80-016588ed60d9 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:18.838555 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-e6536b5c-07f8-4037-ad80-016588ed60d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:18.910617 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 955/1925] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:18.920845 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d66d3-65b3-4b67-980a-11c0519543fb admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:18.922490 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-a24d66d3-65b3-4b67-980a-11c0519543fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:18.996790 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d66d3-65b3-4b67-980a-11c0519543fb admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:19.001318 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398ca399-3121-491e-8539-fdb936c39610 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:19.002648 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ca399-3121-491e-8539-fdb936c39610 service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:19.003375 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ca399-3121-491e-8539-fdb936c39610 service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:19.003776 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ca399-3121-491e-8539-fdb936c39610 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:19.016581 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398ca399-3121-491e-8539-fdb936c39610 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:19.018056 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398ca399-3121-491e-8539-fdb936c39610 service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:19.030711 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 971/1926] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:57:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:57:19.328309 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6536b5c-07f8-4037-ad80-016588ed60d9 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:19.333074 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-e6536b5c-07f8-4037-ad80-016588ed60d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:19.351833 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6536b5c-07f8-4037-ad80-016588ed60d9 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:19.358790 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf53218-d379-43e3-8e99-b28eb4801571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:19.359957 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf53218-d379-43e3-8e99-b28eb4801571 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:19.360284 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf53218-d379-43e3-8e99-b28eb4801571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:19.360560 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf53218-d379-43e3-8e99-b28eb4801571 admin admin] PATH_INFO: `/v3/projects/ba26dcd960594370b237a9adf3b8f7d6` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:19.361887 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf53218-d379-43e3-8e99-b28eb4801571 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba26dcd960594370b237a9adf3b8f7d6)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:19.364692 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf53218-d379-43e3-8e99-b28eb4801571 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:19.430981 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-cdf53218-d379-43e3-8e99-b28eb4801571 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba26dcd960594370b237a9adf3b8f7d6', 'request_id': 'req-cdf53218-d379-43e3-8e99-b28eb4801571'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:19.441003 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-cdf53218-d379-43e3-8e99-b28eb4801571 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba26dcd960594370b237a9adf3b8f7d6', 'request_id': 'req-cdf53218-d379-43e3-8e99-b28eb4801571'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:19.455842 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-cdf53218-d379-43e3-8e99-b28eb4801571 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba26dcd960594370b237a9adf3b8f7d6', 'request_id': 'req-cdf53218-d379-43e3-8e99-b28eb4801571'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:19.467647 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-cdf53218-d379-43e3-8e99-b28eb4801571 admin admin] The token cache is being invalidate because project ba26dcd960594370b237a9adf3b8f7d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:57:19.467941 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-cdf53218-d379-43e3-8e99-b28eb4801571 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:19.468597 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-cdf53218-d379-43e3-8e99-b28eb4801571 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:19.470627 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 956/1927] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:57:19 2022] DELETE /identity/v3/projects/ba26dcd960594370b237a9adf3b8f7d6 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:19.474760 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398ca399-3121-491e-8539-fdb936c39610 service neutron] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:19.484468 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398ca399-3121-491e-8539-fdb936c39610 service neutron] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:19.549744 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2811d7-f2b8-4463-bd34-fa270b8932ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:19.550609 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2811d7-f2b8-4463-bd34-fa270b8932ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:19.550907 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2811d7-f2b8-4463-bd34-fa270b8932ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:19.551169 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2811d7-f2b8-4463-bd34-fa270b8932ff admin admin] PATH_INFO: `/v3/users/a31eb86c055a4e959286f37c594a0019` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:19.552605 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2811d7-f2b8-4463-bd34-fa270b8932ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=a31eb86c055a4e959286f37c594a0019)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:19.554554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2811d7-f2b8-4463-bd34-fa270b8932ff admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:19.625361 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-de2811d7-f2b8-4463-bd34-fa270b8932ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a31eb86c055a4e959286f37c594a0019'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:19.637761 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-de2811d7-f2b8-4463-bd34-fa270b8932ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a31eb86c055a4e959286f37c594a0019'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:19.646404 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-de2811d7-f2b8-4463-bd34-fa270b8932ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a31eb86c055a4e959286f37c594a0019'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:19.657563 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-de2811d7-f2b8-4463-bd34-fa270b8932ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a31eb86c055a4e959286f37c594a0019'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:19.658254 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-de2811d7-f2b8-4463-bd34-fa270b8932ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a31eb86c055a4e959286f37c594a0019'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:19.664519 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 972/1928] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:57:19 2022] DELETE /identity/v3/users/a31eb86c055a4e959286f37c594a0019 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:20.269554 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf53218-d379-43e3-8e99-b28eb4801571 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:20.273864 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-cdf53218-d379-43e3-8e99-b28eb4801571 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:20.313875 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf53218-d379-43e3-8e99-b28eb4801571 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:20.371028 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:20.476909 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c74608-f649-423c-834a-14beb9d8ea76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:20.478044 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c74608-f649-423c-834a-14beb9d8ea76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:20.480679 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c74608-f649-423c-834a-14beb9d8ea76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:20.481140 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c74608-f649-423c-834a-14beb9d8ea76 admin admin] PATH_INFO: `/v3/projects/3044ee619657457f8fd7734fdf86e7bf` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:20.483374 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c74608-f649-423c-834a-14beb9d8ea76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3044ee619657457f8fd7734fdf86e7bf)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:20.486140 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c74608-f649-423c-834a-14beb9d8ea76 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:20.620711 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-69c74608-f649-423c-834a-14beb9d8ea76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3044ee619657457f8fd7734fdf86e7bf', 'request_id': 'req-69c74608-f649-423c-834a-14beb9d8ea76'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:20.639100 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-69c74608-f649-423c-834a-14beb9d8ea76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3044ee619657457f8fd7734fdf86e7bf', 'request_id': 'req-69c74608-f649-423c-834a-14beb9d8ea76'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:20.660524 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-69c74608-f649-423c-834a-14beb9d8ea76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3044ee619657457f8fd7734fdf86e7bf', 'request_id': 'req-69c74608-f649-423c-834a-14beb9d8ea76'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:20.680031 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-69c74608-f649-423c-834a-14beb9d8ea76 admin admin] The token cache is being invalidate because project 3044ee619657457f8fd7734fdf86e7bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:57:20.680323 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-69c74608-f649-423c-834a-14beb9d8ea76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:20.684668 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-69c74608-f649-423c-834a-14beb9d8ea76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:20.687836 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 957/1929] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:57:20 2022] DELETE /identity/v3/projects/3044ee619657457f8fd7734fdf86e7bf => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:20.692994 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3cbb61-b3df-4981-ab9e-4386aa93e2dc None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:20.693488 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3cbb61-b3df-4981-ab9e-4386aa93e2dc None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:20.693488 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3cbb61-b3df-4981-ab9e-4386aa93e2dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:20.713368 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:20.733231 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c74608-f649-423c-834a-14beb9d8ea76 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:20.734026 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-69c74608-f649-423c-834a-14beb9d8ea76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:20.745623 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-ae3cbb61-b3df-4981-ab9e-4386aa93e2dc None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:20.747829 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-ae3cbb61-b3df-4981-ab9e-4386aa93e2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:20.916041 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c74608-f649-423c-834a-14beb9d8ea76 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:20.929693 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccef250-87c1-47b0-99d5-a3914259e489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:20.930915 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccef250-87c1-47b0-99d5-a3914259e489 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:20.933950 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccef250-87c1-47b0-99d5-a3914259e489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:20.934140 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccef250-87c1-47b0-99d5-a3914259e489 admin admin] PATH_INFO: `/v3/users/559cc0b000414c159653f9625b4fc3e1` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:20.935864 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccef250-87c1-47b0-99d5-a3914259e489 admin admin] RBAC: Authorizing `identity:delete_user(user_id=559cc0b000414c159653f9625b4fc3e1)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:20.936646 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 973/1930] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 01:57:20.938480 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccef250-87c1-47b0-99d5-a3914259e489 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:20.945667 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de00ca-53c5-4553-83e7-8292d1e05516 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:20.946044 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de00ca-53c5-4553-83e7-8292d1e05516 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:20.946325 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de00ca-53c5-4553-83e7-8292d1e05516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:20.981239 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-61de00ca-53c5-4553-83e7-8292d1e05516 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:20.984977 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-61de00ca-53c5-4553-83e7-8292d1e05516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:21.006860 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 974/1931] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:21.008957 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de00ca-53c5-4553-83e7-8292d1e05516 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:21.026208 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-bccef250-87c1-47b0-99d5-a3914259e489 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '559cc0b000414c159653f9625b4fc3e1'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:21.040065 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:21.045831 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-bccef250-87c1-47b0-99d5-a3914259e489 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '559cc0b000414c159653f9625b4fc3e1'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:21.051575 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de00ca-53c5-4553-83e7-8292d1e05516 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:21.060914 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a87029-2ad3-466d-97bb-a2259b05313d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:21.061974 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a87029-2ad3-466d-97bb-a2259b05313d admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:21.065382 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a87029-2ad3-466d-97bb-a2259b05313d admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:21.065662 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a87029-2ad3-466d-97bb-a2259b05313d admin admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:21.066438 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a87029-2ad3-466d-97bb-a2259b05313d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:57:21.066536 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-bccef250-87c1-47b0-99d5-a3914259e489 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '559cc0b000414c159653f9625b4fc3e1'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:21.066837 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a87029-2ad3-466d-97bb-a2259b05313d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:21.072342 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a87029-2ad3-466d-97bb-a2259b05313d admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:21.077129 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-bccef250-87c1-47b0-99d5-a3914259e489 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '559cc0b000414c159653f9625b4fc3e1'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:21.077943 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-bccef250-87c1-47b0-99d5-a3914259e489 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '559cc0b000414c159653f9625b4fc3e1'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:21.087277 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 958/1932] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:57:20 2022] DELETE /identity/v3/users/559cc0b000414c159653f9625b4fc3e1 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:21.091300 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 975/1933] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 01:57:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:57:21.092112 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911c6e8-0367-4a34-b6f3-71cf2dfbc90e None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:21.092510 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911c6e8-0367-4a34-b6f3-71cf2dfbc90e None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:21.094537 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911c6e8-0367-4a34-b6f3-71cf2dfbc90e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:21.096478 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a87029-2ad3-466d-97bb-a2259b05313d admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:21.105710 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-b4a87029-2ad3-466d-97bb-a2259b05313d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:21.117352 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-6911c6e8-0367-4a34-b6f3-71cf2dfbc90e None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:21.119029 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-6911c6e8-0367-4a34-b6f3-71cf2dfbc90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:21.124992 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a87029-2ad3-466d-97bb-a2259b05313d admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:21.227066 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e92d9-15fe-4ff4-a828-d750d180b814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:21.228251 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e92d9-15fe-4ff4-a828-d750d180b814 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:21.228532 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e92d9-15fe-4ff4-a828-d750d180b814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:21.228804 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e92d9-15fe-4ff4-a828-d750d180b814 admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:21.229498 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9e92d9-15fe-4ff4-a828-d750d180b814 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:21.231428 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9e92d9-15fe-4ff4-a828-d750d180b814 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:21.253429 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 959/1934] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:21.266903 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911c6e8-0367-4a34-b6f3-71cf2dfbc90e None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:21.272089 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-6911c6e8-0367-4a34-b6f3-71cf2dfbc90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:21.307601 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 976/1935] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:57:21 2022] POST /identity/v3/projects => generated 383 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:57:21.309292 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e92d9-15fe-4ff4-a828-d750d180b814 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:21.414322 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911c6e8-0367-4a34-b6f3-71cf2dfbc90e None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:21.450779 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e92d9-15fe-4ff4-a828-d750d180b814 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:21.457099 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb80e0-2631-4564-9e92-203afba8cda3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:21.458065 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb80e0-2631-4564-9e92-203afba8cda3 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:21.459104 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb80e0-2631-4564-9e92-203afba8cda3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:21.459591 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb80e0-2631-4564-9e92-203afba8cda3 admin admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:21.460542 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eb80e0-2631-4564-9e92-203afba8cda3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:21.462871 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eb80e0-2631-4564-9e92-203afba8cda3 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:21.490571 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 977/1936] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:57:21 2022] POST /identity/v3/users => generated 292 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:57:21.496848 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb80e0-2631-4564-9e92-203afba8cda3 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:21.504241 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:21.514262 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d6156-fcee-4598-a0b6-9b1152396bbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:21.514404 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb80e0-2631-4564-9e92-203afba8cda3 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:21.515770 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d6156-fcee-4598-a0b6-9b1152396bbd service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:21.516373 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d6156-fcee-4598-a0b6-9b1152396bbd service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:21.516797 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d6156-fcee-4598-a0b6-9b1152396bbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:21.528950 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d35677-21d7-45ab-970e-849ee410858d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:21.531415 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84d6156-fcee-4598-a0b6-9b1152396bbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:21.531693 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d35677-21d7-45ab-970e-849ee410858d admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:21.532631 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d35677-21d7-45ab-970e-849ee410858d admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:21.533062 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d35677-21d7-45ab-970e-849ee410858d admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:21.535577 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d35677-21d7-45ab-970e-849ee410858d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:21.536359 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84d6156-fcee-4598-a0b6-9b1152396bbd service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:21.538689 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d35677-21d7-45ab-970e-849ee410858d admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:21.567339 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 960/1937] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:57:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 01:57:21.567437 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 978/1938] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:57:21 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:57:21.567753 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d6156-fcee-4598-a0b6-9b1152396bbd service neutron] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:21.580912 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d6156-fcee-4598-a0b6-9b1152396bbd service neutron] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:21.588977 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8098293e-b5c8-4ccf-b04c-b63bcfb91d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:21.589652 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8098293e-b5c8-4ccf-b04c-b63bcfb91d07 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:21.589931 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8098293e-b5c8-4ccf-b04c-b63bcfb91d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:21.590811 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8098293e-b5c8-4ccf-b04c-b63bcfb91d07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:21.591595 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8098293e-b5c8-4ccf-b04c-b63bcfb91d07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:21.593189 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8098293e-b5c8-4ccf-b04c-b63bcfb91d07 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:21.604280 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 961/1939] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:57:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 01:57:21.608295 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d35677-21d7-45ab-970e-849ee410858d admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:21.618676 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d35677-21d7-45ab-970e-849ee410858d admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:21.624115 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bad611-0b24-41a5-8f96-6fa9089f9eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:21.625077 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bad611-0b24-41a5-8f96-6fa9089f9eb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:21.625307 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bad611-0b24-41a5-8f96-6fa9089f9eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:21.625612 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bad611-0b24-41a5-8f96-6fa9089f9eb0 admin admin] PATH_INFO: `/v3/projects/1fcb16e5774042bb869cdc085dd9b057/users/aaad16e4b7f54ec88e5a6894f99e1b78/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:21.643535 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bad611-0b24-41a5-8f96-6fa9089f9eb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fcb16e5774042bb869cdc085dd9b057, user_id=aaad16e4b7f54ec88e5a6894f99e1b78, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:21.646649 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bad611-0b24-41a5-8f96-6fa9089f9eb0 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:21.662485 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 979/1940] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:57:21 2022] PUT /identity/v3/projects/1fcb16e5774042bb869cdc085dd9b057/users/aaad16e4b7f54ec88e5a6894f99e1b78/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 01:57:21.673130 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8b554-a781-4b4c-9147-cec2ed391b28 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:21.673523 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8b554-a781-4b4c-9147-cec2ed391b28 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:21.674219 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8b554-a781-4b4c-9147-cec2ed391b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:21.703082 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-b0c8b554-a781-4b4c-9147-cec2ed391b28 None None] MFA Rules not processed for user `aaad16e4b7f54ec88e5a6894f99e1b78`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:21.704407 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-b0c8b554-a781-4b4c-9147-cec2ed391b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:21.717398 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:21.780250 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 962/1941] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 01:57:21.784966 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bad611-0b24-41a5-8f96-6fa9089f9eb0 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:21.855162 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bad611-0b24-41a5-8f96-6fa9089f9eb0 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:21.861694 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c6125-76fd-420a-9c80-0043c9c3f7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:21.862497 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c6125-76fd-420a-9c80-0043c9c3f7b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:21.862785 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c6125-76fd-420a-9c80-0043c9c3f7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:21.863062 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c6125-76fd-420a-9c80-0043c9c3f7b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:21.864005 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49c6125-76fd-420a-9c80-0043c9c3f7b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:21.865793 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49c6125-76fd-420a-9c80-0043c9c3f7b1 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:21.871078 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c8b554-a781-4b4c-9147-cec2ed391b28 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:21.872504 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-b0c8b554-a781-4b4c-9147-cec2ed391b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:21.887046 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c8b554-a781-4b4c-9147-cec2ed391b28 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:21.892912 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151c1190-7fec-4e15-92ac-9dd3a11a6c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:21.893845 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151c1190-7fec-4e15-92ac-9dd3a11a6c99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:21.894136 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151c1190-7fec-4e15-92ac-9dd3a11a6c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:21.894423 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151c1190-7fec-4e15-92ac-9dd3a11a6c99 admin admin] PATH_INFO: `/v3/projects/e8f79e76a31742019a5e1a8ce805106f` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:21.895747 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151c1190-7fec-4e15-92ac-9dd3a11a6c99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8f79e76a31742019a5e1a8ce805106f)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:21.897735 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151c1190-7fec-4e15-92ac-9dd3a11a6c99 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:21.901913 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 980/1942] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:57:21 2022] POST /identity/v3/projects => generated 381 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 01:57:21.905798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c6125-76fd-420a-9c80-0043c9c3f7b1 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:21.964996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-151c1190-7fec-4e15-92ac-9dd3a11a6c99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8f79e76a31742019a5e1a8ce805106f', 'request_id': 'req-151c1190-7fec-4e15-92ac-9dd3a11a6c99'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:21.971325 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-151c1190-7fec-4e15-92ac-9dd3a11a6c99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8f79e76a31742019a5e1a8ce805106f', 'request_id': 'req-151c1190-7fec-4e15-92ac-9dd3a11a6c99'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:21.980980 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-151c1190-7fec-4e15-92ac-9dd3a11a6c99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8f79e76a31742019a5e1a8ce805106f', 'request_id': 'req-151c1190-7fec-4e15-92ac-9dd3a11a6c99'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:21.982816 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c6125-76fd-420a-9c80-0043c9c3f7b1 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:21.995324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-151c1190-7fec-4e15-92ac-9dd3a11a6c99 admin admin] The token cache is being invalidate because project e8f79e76a31742019a5e1a8ce805106f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:57:21.995616 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-151c1190-7fec-4e15-92ac-9dd3a11a6c99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:21.996280 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-151c1190-7fec-4e15-92ac-9dd3a11a6c99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:21.997879 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 963/1943] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:57:21 2022] DELETE /identity/v3/projects/e8f79e76a31742019a5e1a8ce805106f => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 01:57:22.001681 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151c1190-7fec-4e15-92ac-9dd3a11a6c99 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:22.003506 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-151c1190-7fec-4e15-92ac-9dd3a11a6c99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:22.053738 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635a1690-0812-404d-a913-230673924bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:22.054783 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a1690-0812-404d-a913-230673924bc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:22.054783 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a1690-0812-404d-a913-230673924bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:22.054988 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a1690-0812-404d-a913-230673924bc1 admin admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:22.055667 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635a1690-0812-404d-a913-230673924bc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:22.058908 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635a1690-0812-404d-a913-230673924bc1 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:22.066540 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:22.090166 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151c1190-7fec-4e15-92ac-9dd3a11a6c99 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:22.106433 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9fca3b-d6d8-4a73-8b02-cc47f11bd9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:22.107983 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9fca3b-d6d8-4a73-8b02-cc47f11bd9a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:22.112446 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9fca3b-d6d8-4a73-8b02-cc47f11bd9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:22.112446 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9fca3b-d6d8-4a73-8b02-cc47f11bd9a2 admin admin] PATH_INFO: `/v3/users/7b9053c8f3f24810a8b96601193d7f9f` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:22.112718 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 981/1944] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:57:21 2022] POST /identity/v3/users => generated 297 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 01:57:22.113314 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9fca3b-d6d8-4a73-8b02-cc47f11bd9a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b9053c8f3f24810a8b96601193d7f9f)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:22.113703 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635a1690-0812-404d-a913-230673924bc1 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:22.115837 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9fca3b-d6d8-4a73-8b02-cc47f11bd9a2 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:22.117024 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-635a1690-0812-404d-a913-230673924bc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:22.136002 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635a1690-0812-404d-a913-230673924bc1 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:22.144061 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f786be0b-7fab-4ef2-99b8-6b271b896dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:22.145059 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f786be0b-7fab-4ef2-99b8-6b271b896dc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:22.145826 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f786be0b-7fab-4ef2-99b8-6b271b896dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:22.146473 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f786be0b-7fab-4ef2-99b8-6b271b896dc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:22.147568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f786be0b-7fab-4ef2-99b8-6b271b896dc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:22.149836 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f786be0b-7fab-4ef2-99b8-6b271b896dc3 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:22.172770 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 982/1945] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:57:22 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:57:22.177458 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f786be0b-7fab-4ef2-99b8-6b271b896dc3 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:22.183234 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:22.194348 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f786be0b-7fab-4ef2-99b8-6b271b896dc3 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:22.197681 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-0b9fca3b-d6d8-4a73-8b02-cc47f11bd9a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b9053c8f3f24810a8b96601193d7f9f'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:22.202560 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46598e5-41a9-45f8-ad6a-906988ab0f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:22.203577 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46598e5-41a9-45f8-ad6a-906988ab0f15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:22.203858 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46598e5-41a9-45f8-ad6a-906988ab0f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:22.204144 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46598e5-41a9-45f8-ad6a-906988ab0f15 admin admin] PATH_INFO: `/v3/projects/7c8e7eaa8a624912b74063acb9dc909a/users/b6ccae2e1c0c4545877ae09ada78829e/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:22.218762 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-0b9fca3b-d6d8-4a73-8b02-cc47f11bd9a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b9053c8f3f24810a8b96601193d7f9f'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:22.226644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46598e5-41a9-45f8-ad6a-906988ab0f15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c8e7eaa8a624912b74063acb9dc909a, user_id=b6ccae2e1c0c4545877ae09ada78829e, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:22.229212 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-0b9fca3b-d6d8-4a73-8b02-cc47f11bd9a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b9053c8f3f24810a8b96601193d7f9f'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:22.235763 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46598e5-41a9-45f8-ad6a-906988ab0f15 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:22.243655 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-0b9fca3b-d6d8-4a73-8b02-cc47f11bd9a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b9053c8f3f24810a8b96601193d7f9f'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:22.245115 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-0b9fca3b-d6d8-4a73-8b02-cc47f11bd9a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b9053c8f3f24810a8b96601193d7f9f'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:22.256228 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 964/1946] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:57:22 2022] DELETE /identity/v3/users/7b9053c8f3f24810a8b96601193d7f9f => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:22.274457 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 983/1947] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:57:22 2022] PUT /identity/v3/projects/7c8e7eaa8a624912b74063acb9dc909a/users/b6ccae2e1c0c4545877ae09ada78829e/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 01:57:22.280259 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9fca3b-d6d8-4a73-8b02-cc47f11bd9a2 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:22.289175 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-0b9fca3b-d6d8-4a73-8b02-cc47f11bd9a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:22.385303 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9fca3b-d6d8-4a73-8b02-cc47f11bd9a2 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:22.391812 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047cdeb8-a50a-459c-9095-02122cf2da81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:22.392203 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047cdeb8-a50a-459c-9095-02122cf2da81 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:22.392615 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047cdeb8-a50a-459c-9095-02122cf2da81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:22.392880 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047cdeb8-a50a-459c-9095-02122cf2da81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:22.393622 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047cdeb8-a50a-459c-9095-02122cf2da81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:22.395392 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047cdeb8-a50a-459c-9095-02122cf2da81 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:22.407366 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 965/1948] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:57:22 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:57:22.411809 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46598e5-41a9-45f8-ad6a-906988ab0f15 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:22.509978 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46598e5-41a9-45f8-ad6a-906988ab0f15 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:22.514852 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c487f-ee47-4f6c-8b1e-9641d7b8591d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:22.515795 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c487f-ee47-4f6c-8b1e-9641d7b8591d admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:22.516075 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c487f-ee47-4f6c-8b1e-9641d7b8591d admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:22.516348 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c487f-ee47-4f6c-8b1e-9641d7b8591d admin admin] PATH_INFO: `/v3/projects/7c8e7eaa8a624912b74063acb9dc909a/users/b6ccae2e1c0c4545877ae09ada78829e/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:22.518656 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5c487f-ee47-4f6c-8b1e-9641d7b8591d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c8e7eaa8a624912b74063acb9dc909a, user_id=b6ccae2e1c0c4545877ae09ada78829e, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:22.522300 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5c487f-ee47-4f6c-8b1e-9641d7b8591d admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:22.535445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 984/1949] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:57:22 2022] PUT /identity/v3/projects/7c8e7eaa8a624912b74063acb9dc909a/users/b6ccae2e1c0c4545877ae09ada78829e/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:22.546150 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e6c50-09e6-431a-bfd4-2cc3f9dade12 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:22.549800 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e6c50-09e6-431a-bfd4-2cc3f9dade12 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:22.550330 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e6c50-09e6-431a-bfd4-2cc3f9dade12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:22.626714 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-887e6c50-09e6-431a-bfd4-2cc3f9dade12 None None] MFA Rules not processed for user `b6ccae2e1c0c4545877ae09ada78829e`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:22.632101 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-887e6c50-09e6-431a-bfd4-2cc3f9dade12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:22.865093 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 966/1950] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:22 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:22.878474 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c487f-ee47-4f6c-8b1e-9641d7b8591d admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:22.880501 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-0b5c487f-ee47-4f6c-8b1e-9641d7b8591d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:22.963408 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c487f-ee47-4f6c-8b1e-9641d7b8591d admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:22.970881 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebc14e1-d294-492f-974a-020edd1e5c20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:22.971592 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc14e1-d294-492f-974a-020edd1e5c20 service cinder] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:22.972061 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc14e1-d294-492f-974a-020edd1e5c20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:22.972445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc14e1-d294-492f-974a-020edd1e5c20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:22.983439 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebc14e1-d294-492f-974a-020edd1e5c20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:22.985037 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebc14e1-d294-492f-974a-020edd1e5c20 service cinder] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:22.995883 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 985/1951] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:57:22 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:57:23.096199 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887e6c50-09e6-431a-bfd4-2cc3f9dade12 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:23.098552 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-887e6c50-09e6-431a-bfd4-2cc3f9dade12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:23.127775 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebc14e1-d294-492f-974a-020edd1e5c20 service cinder] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:23.129506 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-8ebc14e1-d294-492f-974a-020edd1e5c20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:23.147256 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebc14e1-d294-492f-974a-020edd1e5c20 service cinder] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:23.163086 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5333ec93-6adf-4465-8450-ded4467917a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:23.165046 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5333ec93-6adf-4465-8450-ded4467917a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:23.165487 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5333ec93-6adf-4465-8450-ded4467917a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:23.165915 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5333ec93-6adf-4465-8450-ded4467917a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:23.168262 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-5333ec93-6adf-4465-8450-ded4467917a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:23.173458 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:23.207572 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887e6c50-09e6-431a-bfd4-2cc3f9dade12 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:23.213333 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6591089-b369-4f10-b5e9-7f6700970226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:23.214258 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6591089-b369-4f10-b5e9-7f6700970226 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:23.214553 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6591089-b369-4f10-b5e9-7f6700970226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:23.214802 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6591089-b369-4f10-b5e9-7f6700970226 admin admin] PATH_INFO: `/v3/users/6828d54b02ad497d80e57f9b0f4fbe95` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:23.216067 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6591089-b369-4f10-b5e9-7f6700970226 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6828d54b02ad497d80e57f9b0f4fbe95)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:23.218001 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6591089-b369-4f10-b5e9-7f6700970226 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:23.226343 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:23.239461 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5333ec93-6adf-4465-8450-ded4467917a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:23.240940 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5333ec93-6adf-4465-8450-ded4467917a1 service cinder] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:23.250946 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 986/1952] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:57:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:57:23.251762 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5333ec93-6adf-4465-8450-ded4467917a1 service cinder] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:23.253128 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-5333ec93-6adf-4465-8450-ded4467917a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:23.267947 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5333ec93-6adf-4465-8450-ded4467917a1 service cinder] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:23.272886 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad444a03-5461-47d1-96a8-dfb4479a85ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:23.273732 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad444a03-5461-47d1-96a8-dfb4479a85ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:23.274013 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad444a03-5461-47d1-96a8-dfb4479a85ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:23.274289 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad444a03-5461-47d1-96a8-dfb4479a85ff admin admin] PATH_INFO: `/v3/projects/47edb9244af243ac8ce34bb543bc319b` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:23.275423 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-f6591089-b369-4f10-b5e9-7f6700970226 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6828d54b02ad497d80e57f9b0f4fbe95'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:23.275625 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad444a03-5461-47d1-96a8-dfb4479a85ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=47edb9244af243ac8ce34bb543bc319b)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:23.277872 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad444a03-5461-47d1-96a8-dfb4479a85ff admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:23.299871 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-f6591089-b369-4f10-b5e9-7f6700970226 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6828d54b02ad497d80e57f9b0f4fbe95'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:23.314853 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-f6591089-b369-4f10-b5e9-7f6700970226 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6828d54b02ad497d80e57f9b0f4fbe95'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:23.321862 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-f6591089-b369-4f10-b5e9-7f6700970226 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6828d54b02ad497d80e57f9b0f4fbe95'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:23.322881 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-f6591089-b369-4f10-b5e9-7f6700970226 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6828d54b02ad497d80e57f9b0f4fbe95'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:23.335675 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 967/1953] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:57:23 2022] DELETE /identity/v3/users/6828d54b02ad497d80e57f9b0f4fbe95 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:23.337848 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9905a89a-01c2-4a08-b54e-427016f9f1ae None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:23.338148 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9905a89a-01c2-4a08-b54e-427016f9f1ae None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:23.338429 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9905a89a-01c2-4a08-b54e-427016f9f1ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:23.370107 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-9905a89a-01c2-4a08-b54e-427016f9f1ae None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:23.373268 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-ad444a03-5461-47d1-96a8-dfb4479a85ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47edb9244af243ac8ce34bb543bc319b', 'request_id': 'req-ad444a03-5461-47d1-96a8-dfb4479a85ff'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:23.373401 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-9905a89a-01c2-4a08-b54e-427016f9f1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:23.381391 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-ad444a03-5461-47d1-96a8-dfb4479a85ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47edb9244af243ac8ce34bb543bc319b', 'request_id': 'req-ad444a03-5461-47d1-96a8-dfb4479a85ff'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:23.393887 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-ad444a03-5461-47d1-96a8-dfb4479a85ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47edb9244af243ac8ce34bb543bc319b', 'request_id': 'req-ad444a03-5461-47d1-96a8-dfb4479a85ff'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:23.410974 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-ad444a03-5461-47d1-96a8-dfb4479a85ff admin admin] The token cache is being invalidate because project 47edb9244af243ac8ce34bb543bc319b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:57:23.411433 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-ad444a03-5461-47d1-96a8-dfb4479a85ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:23.412753 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-ad444a03-5461-47d1-96a8-dfb4479a85ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:23.414967 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 987/1954] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:57:23 2022] DELETE /identity/v3/projects/47edb9244af243ac8ce34bb543bc319b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:23.423030 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b152913-2707-4528-9813-c3bf6286cd5b None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:23.424290 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b152913-2707-4528-9813-c3bf6286cd5b None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:23.424715 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b152913-2707-4528-9813-c3bf6286cd5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:23.454286 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-5b152913-2707-4528-9813-c3bf6286cd5b None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:23.455612 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-5b152913-2707-4528-9813-c3bf6286cd5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:23.475917 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 968/1955] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:23.485610 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9905a89a-01c2-4a08-b54e-427016f9f1ae None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:23.488069 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-9905a89a-01c2-4a08-b54e-427016f9f1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:23.568301 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 988/1956] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:23.576795 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c52299d-6e84-4d97-8e58-27be702ff508 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:23.577243 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c52299d-6e84-4d97-8e58-27be702ff508 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:23.577610 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c52299d-6e84-4d97-8e58-27be702ff508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:23.580874 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9905a89a-01c2-4a08-b54e-427016f9f1ae None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:23.585386 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55119cb3-1a32-4e55-9207-17262684186d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:23.586275 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55119cb3-1a32-4e55-9207-17262684186d service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:23.586626 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55119cb3-1a32-4e55-9207-17262684186d service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:23.586971 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55119cb3-1a32-4e55-9207-17262684186d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:23.599975 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55119cb3-1a32-4e55-9207-17262684186d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:23.601815 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55119cb3-1a32-4e55-9207-17262684186d service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:23.604283 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-4c52299d-6e84-4d97-8e58-27be702ff508 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:23.605907 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-4c52299d-6e84-4d97-8e58-27be702ff508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:23.618246 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 989/1957] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:23.620392 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:23.623447 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c52299d-6e84-4d97-8e58-27be702ff508 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:23.641314 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 969/1958] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:57:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 01:57:23.647083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c52299d-6e84-4d97-8e58-27be702ff508 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:23.659383 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14daeb27-bb7c-4213-aeda-14ed1f0ec38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:23.661853 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14daeb27-bb7c-4213-aeda-14ed1f0ec38c admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:23.662138 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14daeb27-bb7c-4213-aeda-14ed1f0ec38c admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:23.662537 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14daeb27-bb7c-4213-aeda-14ed1f0ec38c admin admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:23.663334 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14daeb27-bb7c-4213-aeda-14ed1f0ec38c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:57:23.666335 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14daeb27-bb7c-4213-aeda-14ed1f0ec38c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:23.673294 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14daeb27-bb7c-4213-aeda-14ed1f0ec38c admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:23.694831 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 990/1959] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 01:57:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:57:23.700507 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55119cb3-1a32-4e55-9207-17262684186d service neutron] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:23.711660 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55119cb3-1a32-4e55-9207-17262684186d service neutron] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:23.717282 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6d5f14-538a-4b2e-a485-d130d1d979db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:23.718163 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d5f14-538a-4b2e-a485-d130d1d979db admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:23.718515 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d5f14-538a-4b2e-a485-d130d1d979db admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:23.719114 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d5f14-538a-4b2e-a485-d130d1d979db admin admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:23.720054 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6d5f14-538a-4b2e-a485-d130d1d979db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:23.722187 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6d5f14-538a-4b2e-a485-d130d1d979db admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:23.759015 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 970/1960] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:57:23 2022] POST /identity/v3/projects => generated 381 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:57:23.760516 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14daeb27-bb7c-4213-aeda-14ed1f0ec38c admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:23.771069 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14daeb27-bb7c-4213-aeda-14ed1f0ec38c admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:23.840195 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:23.853216 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021740ee-6874-4151-af4e-913808e7fa14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:23.854063 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021740ee-6874-4151-af4e-913808e7fa14 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:23.854370 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021740ee-6874-4151-af4e-913808e7fa14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:23.854653 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021740ee-6874-4151-af4e-913808e7fa14 admin admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:23.855444 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021740ee-6874-4151-af4e-913808e7fa14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:23.857945 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021740ee-6874-4151-af4e-913808e7fa14 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:23.883167 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 991/1961] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:57:23 2022] POST /identity/v3/users => generated 291 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:57:23.885152 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6d5f14-538a-4b2e-a485-d130d1d979db admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:23.895885 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6d5f14-538a-4b2e-a485-d130d1d979db admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:23.900841 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0ce704-c35f-4ee9-9236-2fb279393156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:23.901492 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ce704-c35f-4ee9-9236-2fb279393156 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:23.901777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ce704-c35f-4ee9-9236-2fb279393156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:23.902052 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ce704-c35f-4ee9-9236-2fb279393156 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:23.902780 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0ce704-c35f-4ee9-9236-2fb279393156 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:23.905045 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0ce704-c35f-4ee9-9236-2fb279393156 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:23.917306 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 971/1962] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:57:23.921516 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021740ee-6874-4151-af4e-913808e7fa14 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:23.932954 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021740ee-6874-4151-af4e-913808e7fa14 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:23.938239 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb246997-03c1-4169-9211-34201d6fcb30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:23.938903 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb246997-03c1-4169-9211-34201d6fcb30 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:23.939182 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb246997-03c1-4169-9211-34201d6fcb30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:23.939494 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb246997-03c1-4169-9211-34201d6fcb30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:23.940224 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb246997-03c1-4169-9211-34201d6fcb30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:23.942238 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb246997-03c1-4169-9211-34201d6fcb30 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:23.952835 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 992/1963] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:57:23.957149 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0ce704-c35f-4ee9-9236-2fb279393156 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:23.969939 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0ce704-c35f-4ee9-9236-2fb279393156 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:23.976123 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd1f35f-9dbf-429c-951a-d055c38b6300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:23.976775 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd1f35f-9dbf-429c-951a-d055c38b6300 admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:23.977144 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd1f35f-9dbf-429c-951a-d055c38b6300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:23.977549 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd1f35f-9dbf-429c-951a-d055c38b6300 admin admin] PATH_INFO: `/v3/projects/2f22a5c2b7dd4b4a9b1487dbff55eace/users/5d066742ffbc4c78b40252f6dabe6efb/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:24.002128 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd1f35f-9dbf-429c-951a-d055c38b6300 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f22a5c2b7dd4b4a9b1487dbff55eace, user_id=5d066742ffbc4c78b40252f6dabe6efb, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:24.004895 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd1f35f-9dbf-429c-951a-d055c38b6300 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:24.028480 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 972/1964] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:57:23 2022] PUT /identity/v3/projects/2f22a5c2b7dd4b4a9b1487dbff55eace/users/5d066742ffbc4c78b40252f6dabe6efb/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:24.039961 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e8fb93-2fe2-4bf1-bf59-ed9567cb9e99 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:24.040268 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e8fb93-2fe2-4bf1-bf59-ed9567cb9e99 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:24.040548 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e8fb93-2fe2-4bf1-bf59-ed9567cb9e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:24.069065 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd1f35f-9dbf-429c-951a-d055c38b6300 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:24.071009 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-0bd1f35f-9dbf-429c-951a-d055c38b6300 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:24.072403 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-b4e8fb93-2fe2-4bf1-bf59-ed9567cb9e99 None None] MFA Rules not processed for user `5d066742ffbc4c78b40252f6dabe6efb`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:24.073667 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-b4e8fb93-2fe2-4bf1-bf59-ed9567cb9e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:24.151530 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 993/1965] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:24.155908 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e8fb93-2fe2-4bf1-bf59-ed9567cb9e99 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:24.164815 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd1f35f-9dbf-429c-951a-d055c38b6300 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:24.166638 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e8fb93-2fe2-4bf1-bf59-ed9567cb9e99 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:24.169924 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d9d813-e892-411b-90d7-d88236813ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:24.170844 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d9d813-e892-411b-90d7-d88236813ee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:24.171318 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d9ef6e-c171-4be9-b606-99525619f70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:24.171468 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d9d813-e892-411b-90d7-d88236813ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:24.171637 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d9d813-e892-411b-90d7-d88236813ee3 admin admin] PATH_INFO: `/v3/projects/15facda1b2534a72852d00b61b63bc1c` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:24.171973 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9ef6e-c171-4be9-b606-99525619f70c admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:24.172263 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9ef6e-c171-4be9-b606-99525619f70c admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:24.172527 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d9ef6e-c171-4be9-b606-99525619f70c admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:24.172844 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d9d813-e892-411b-90d7-d88236813ee3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15facda1b2534a72852d00b61b63bc1c)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:24.173217 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d9ef6e-c171-4be9-b606-99525619f70c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:24.175161 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d9d813-e892-411b-90d7-d88236813ee3 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:24.175453 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d9ef6e-c171-4be9-b606-99525619f70c admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:24.213386 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 994/1966] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:57:24 2022] POST /identity/v3/projects => generated 377 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 01:57:24.217683 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d9ef6e-c171-4be9-b606-99525619f70c admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:24.229414 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-40d9d813-e892-411b-90d7-d88236813ee3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15facda1b2534a72852d00b61b63bc1c', 'request_id': 'req-40d9d813-e892-411b-90d7-d88236813ee3'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:24.236264 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-40d9d813-e892-411b-90d7-d88236813ee3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15facda1b2534a72852d00b61b63bc1c', 'request_id': 'req-40d9d813-e892-411b-90d7-d88236813ee3'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:24.245176 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-40d9d813-e892-411b-90d7-d88236813ee3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15facda1b2534a72852d00b61b63bc1c', 'request_id': 'req-40d9d813-e892-411b-90d7-d88236813ee3'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:24.256440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-40d9d813-e892-411b-90d7-d88236813ee3 admin admin] The token cache is being invalidate because project 15facda1b2534a72852d00b61b63bc1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:57:24.256728 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-40d9d813-e892-411b-90d7-d88236813ee3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:24.257379 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-40d9d813-e892-411b-90d7-d88236813ee3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:24.258998 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 973/1967] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:57:24 2022] DELETE /identity/v3/projects/15facda1b2534a72852d00b61b63bc1c => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:24.262992 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d9d813-e892-411b-90d7-d88236813ee3 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:24.264458 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-40d9d813-e892-411b-90d7-d88236813ee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:24.283667 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d9ef6e-c171-4be9-b606-99525619f70c admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:24.303163 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:24.334232 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d9d813-e892-411b-90d7-d88236813ee3 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:24.335804 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:24.339642 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2deba94-9195-4558-85f3-e9738138cf3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:24.340513 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2deba94-9195-4558-85f3-e9738138cf3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:24.340794 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2deba94-9195-4558-85f3-e9738138cf3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:24.341071 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2deba94-9195-4558-85f3-e9738138cf3d admin admin] PATH_INFO: `/v3/users/d824028753bc4e2eb7284c93274eb65a` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:24.342300 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2deba94-9195-4558-85f3-e9738138cf3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d824028753bc4e2eb7284c93274eb65a)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:24.344285 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2deba94-9195-4558-85f3-e9738138cf3d admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:24.349777 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2193a87a-4b94-48e1-a062-07fa2fc38654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:24.350446 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2193a87a-4b94-48e1-a062-07fa2fc38654 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:24.350723 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2193a87a-4b94-48e1-a062-07fa2fc38654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:24.350991 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2193a87a-4b94-48e1-a062-07fa2fc38654 admin admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:24.351728 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2193a87a-4b94-48e1-a062-07fa2fc38654 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:24.353741 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2193a87a-4b94-48e1-a062-07fa2fc38654 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:24.375863 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 995/1968] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:57:24 2022] POST /identity/v3/users => generated 295 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 01:57:24.380651 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2193a87a-4b94-48e1-a062-07fa2fc38654 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:24.382166 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-2193a87a-4b94-48e1-a062-07fa2fc38654 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:24.394775 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2193a87a-4b94-48e1-a062-07fa2fc38654 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:24.399889 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-e2deba94-9195-4558-85f3-e9738138cf3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd824028753bc4e2eb7284c93274eb65a'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:24.409377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f2d4b5-eab0-4401-8fce-6fd9b4e49717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:24.410471 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2d4b5-eab0-4401-8fce-6fd9b4e49717 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:24.410471 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2d4b5-eab0-4401-8fce-6fd9b4e49717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:24.410888 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2d4b5-eab0-4401-8fce-6fd9b4e49717 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:24.411693 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f2d4b5-eab0-4401-8fce-6fd9b4e49717 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:24.413299 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f2d4b5-eab0-4401-8fce-6fd9b4e49717 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:24.420096 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-e2deba94-9195-4558-85f3-e9738138cf3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd824028753bc4e2eb7284c93274eb65a'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:24.424472 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 996/1969] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:57:24 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:57:24.429084 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f2d4b5-eab0-4401-8fce-6fd9b4e49717 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:24.437082 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-e2deba94-9195-4558-85f3-e9738138cf3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd824028753bc4e2eb7284c93274eb65a'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:24.440657 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f2d4b5-eab0-4401-8fce-6fd9b4e49717 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:24.445002 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-e2deba94-9195-4558-85f3-e9738138cf3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd824028753bc4e2eb7284c93274eb65a'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:24.445676 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-e2deba94-9195-4558-85f3-e9738138cf3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd824028753bc4e2eb7284c93274eb65a'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:24.446352 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6659a3fd-49b2-4e2d-9df5-fc61b5b47465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:24.447415 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659a3fd-49b2-4e2d-9df5-fc61b5b47465 admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:24.447761 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659a3fd-49b2-4e2d-9df5-fc61b5b47465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:24.448031 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659a3fd-49b2-4e2d-9df5-fc61b5b47465 admin admin] PATH_INFO: `/v3/projects/58941b5e74fe431687a65fa7c3b77e23/users/52068a13e7db4cf0a2e1f0f4b456bd23/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:24.451766 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 974/1970] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:57:24 2022] DELETE /identity/v3/users/d824028753bc4e2eb7284c93274eb65a => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 01:57:24.463175 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6659a3fd-49b2-4e2d-9df5-fc61b5b47465 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58941b5e74fe431687a65fa7c3b77e23, user_id=52068a13e7db4cf0a2e1f0f4b456bd23, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:24.465864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6659a3fd-49b2-4e2d-9df5-fc61b5b47465 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:24.480518 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 997/1971] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:57:24 2022] PUT /identity/v3/projects/58941b5e74fe431687a65fa7c3b77e23/users/52068a13e7db4cf0a2e1f0f4b456bd23/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 01:57:24.484613 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2deba94-9195-4558-85f3-e9738138cf3d admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:24.486054 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-e2deba94-9195-4558-85f3-e9738138cf3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:24.559194 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2deba94-9195-4558-85f3-e9738138cf3d admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:24.620908 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf40e3b-640a-494a-ba70-00aa39025853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:24.621764 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf40e3b-640a-494a-ba70-00aa39025853 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:24.622154 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf40e3b-640a-494a-ba70-00aa39025853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:24.622529 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf40e3b-640a-494a-ba70-00aa39025853 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:24.623421 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf40e3b-640a-494a-ba70-00aa39025853 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:24.625258 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf40e3b-640a-494a-ba70-00aa39025853 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:24.626695 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:24.639501 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 975/1972] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:57:24 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:57:24.642452 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6659a3fd-49b2-4e2d-9df5-fc61b5b47465 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:24.660236 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6659a3fd-49b2-4e2d-9df5-fc61b5b47465 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:24.670911 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a24888-5c44-4c73-993f-9fa58cc9b80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:24.671842 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a24888-5c44-4c73-993f-9fa58cc9b80f admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:24.672123 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a24888-5c44-4c73-993f-9fa58cc9b80f admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:24.672392 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a24888-5c44-4c73-993f-9fa58cc9b80f admin admin] PATH_INFO: `/v3/projects/58941b5e74fe431687a65fa7c3b77e23/users/52068a13e7db4cf0a2e1f0f4b456bd23/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:24.674679 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a24888-5c44-4c73-993f-9fa58cc9b80f admin admin] RBAC: Authorizing `identity:create_grant(project_id=58941b5e74fe431687a65fa7c3b77e23, user_id=52068a13e7db4cf0a2e1f0f4b456bd23, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:24.677427 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a24888-5c44-4c73-993f-9fa58cc9b80f admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:24.688519 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 998/1973] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:57:24 2022] PUT /identity/v3/projects/58941b5e74fe431687a65fa7c3b77e23/users/52068a13e7db4cf0a2e1f0f4b456bd23/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 01:57:24.699081 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f1ca9e-c564-4e5e-8cf5-07f9e7d13a60 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:24.699413 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f1ca9e-c564-4e5e-8cf5-07f9e7d13a60 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:24.699689 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f1ca9e-c564-4e5e-8cf5-07f9e7d13a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:24.728939 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-a6f1ca9e-c564-4e5e-8cf5-07f9e7d13a60 None None] MFA Rules not processed for user `52068a13e7db4cf0a2e1f0f4b456bd23`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:24.732630 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-a6f1ca9e-c564-4e5e-8cf5-07f9e7d13a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:24.827255 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 976/1974] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:24 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:24.841265 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a24888-5c44-4c73-993f-9fa58cc9b80f admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:24.843404 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-b6a24888-5c44-4c73-993f-9fa58cc9b80f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:24.863198 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f1ca9e-c564-4e5e-8cf5-07f9e7d13a60 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:24.864853 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-a6f1ca9e-c564-4e5e-8cf5-07f9e7d13a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:24.938536 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f1ca9e-c564-4e5e-8cf5-07f9e7d13a60 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:24.951203 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805edfba-60c1-439c-a82e-0303852bc31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:24.952081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a24888-5c44-4c73-993f-9fa58cc9b80f admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:24.953202 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805edfba-60c1-439c-a82e-0303852bc31f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:24.953547 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805edfba-60c1-439c-a82e-0303852bc31f admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:24.953898 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805edfba-60c1-439c-a82e-0303852bc31f admin admin] PATH_INFO: `/v3/projects/2b3fbb7abbad469c81be764a49aa5889` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:24.955724 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805edfba-60c1-439c-a82e-0303852bc31f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b3fbb7abbad469c81be764a49aa5889)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:24.958113 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805edfba-60c1-439c-a82e-0303852bc31f admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:24.958468 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade06401-4086-4c56-b30c-ae96e2d28830 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:24.959126 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade06401-4086-4c56-b30c-ae96e2d28830 service cinder] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:24.959468 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade06401-4086-4c56-b30c-ae96e2d28830 service cinder] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:24.959774 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade06401-4086-4c56-b30c-ae96e2d28830 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:24.971855 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade06401-4086-4c56-b30c-ae96e2d28830 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:24.973300 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade06401-4086-4c56-b30c-ae96e2d28830 service cinder] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:24.985567 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 999/1975] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:57:24 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:57:25.025298 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-805edfba-60c1-439c-a82e-0303852bc31f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b3fbb7abbad469c81be764a49aa5889', 'request_id': 'req-805edfba-60c1-439c-a82e-0303852bc31f'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:25.032768 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-805edfba-60c1-439c-a82e-0303852bc31f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b3fbb7abbad469c81be764a49aa5889', 'request_id': 'req-805edfba-60c1-439c-a82e-0303852bc31f'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:25.052812 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-805edfba-60c1-439c-a82e-0303852bc31f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b3fbb7abbad469c81be764a49aa5889', 'request_id': 'req-805edfba-60c1-439c-a82e-0303852bc31f'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:25.068670 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-805edfba-60c1-439c-a82e-0303852bc31f admin admin] The token cache is being invalidate because project 2b3fbb7abbad469c81be764a49aa5889 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:57:25.069229 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-805edfba-60c1-439c-a82e-0303852bc31f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:25.069782 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-805edfba-60c1-439c-a82e-0303852bc31f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:25.071650 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 977/1976] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:57:24 2022] DELETE /identity/v3/projects/2b3fbb7abbad469c81be764a49aa5889 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:25.078231 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa21cfa-a440-4b90-b1ae-2522d2e17dfe None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:25.078526 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa21cfa-a440-4b90-b1ae-2522d2e17dfe None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:25.078790 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa21cfa-a440-4b90-b1ae-2522d2e17dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:25.095065 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805edfba-60c1-439c-a82e-0303852bc31f admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:25.096879 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-805edfba-60c1-439c-a82e-0303852bc31f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:25.102636 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-dfa21cfa-a440-4b90-b1ae-2522d2e17dfe None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:25.103856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-dfa21cfa-a440-4b90-b1ae-2522d2e17dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:25.111160 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:25.167599 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805edfba-60c1-439c-a82e-0303852bc31f admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:25.171823 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa82be6b-4378-4808-8a5c-4860c997d9b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:25.172522 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa82be6b-4378-4808-8a5c-4860c997d9b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:25.172804 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa82be6b-4378-4808-8a5c-4860c997d9b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:25.173064 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa82be6b-4378-4808-8a5c-4860c997d9b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:25.175616 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-fa82be6b-4378-4808-8a5c-4860c997d9b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:25.176394 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1000/1977] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:25.185025 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68d5f5-2d7f-4138-baef-db3ec0659f66 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:25.185909 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68d5f5-2d7f-4138-baef-db3ec0659f66 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:25.186312 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68d5f5-2d7f-4138-baef-db3ec0659f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:25.218856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-ff68d5f5-2d7f-4138-baef-db3ec0659f66 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:25.218856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-ff68d5f5-2d7f-4138-baef-db3ec0659f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:25.231568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1001/1978] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 01:57:25.233120 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff68d5f5-2d7f-4138-baef-db3ec0659f66 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:25.245299 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff68d5f5-2d7f-4138-baef-db3ec0659f66 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:25.256949 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e6a8b5-c5fe-4f29-9147-42309cafe7aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:25.257685 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6a8b5-c5fe-4f29-9147-42309cafe7aa admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:25.257995 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6a8b5-c5fe-4f29-9147-42309cafe7aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:25.258320 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6a8b5-c5fe-4f29-9147-42309cafe7aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:25.259393 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e6a8b5-c5fe-4f29-9147-42309cafe7aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:57:25.259717 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e6a8b5-c5fe-4f29-9147-42309cafe7aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:25.263919 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e6a8b5-c5fe-4f29-9147-42309cafe7aa admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:25.280064 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1002/1979] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 01:57:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:57:25.281209 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e6a8b5-c5fe-4f29-9147-42309cafe7aa admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:25.301919 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa82be6b-4378-4808-8a5c-4860c997d9b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:25.304070 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa82be6b-4378-4808-8a5c-4860c997d9b6 service cinder] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:25.304702 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e6a8b5-c5fe-4f29-9147-42309cafe7aa admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:25.316000 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 978/1980] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:57:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 01:57:25.320511 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1048f24-262a-4542-b638-a574b790018a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:25.321584 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1048f24-262a-4542-b638-a574b790018a admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:25.322335 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1048f24-262a-4542-b638-a574b790018a admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:25.322801 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1048f24-262a-4542-b638-a574b790018a admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:25.323615 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1048f24-262a-4542-b638-a574b790018a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:25.325585 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1048f24-262a-4542-b638-a574b790018a admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:25.338999 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:25.370894 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1003/1981] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:57:25 2022] POST /identity/v3/projects => generated 377 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:57:25.372416 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa82be6b-4378-4808-8a5c-4860c997d9b6 service cinder] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:25.388531 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa82be6b-4378-4808-8a5c-4860c997d9b6 service cinder] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:25.495350 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cc0d49-04ef-4618-8e18-78556a7f8989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:25.496090 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc0d49-04ef-4618-8e18-78556a7f8989 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:25.496518 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc0d49-04ef-4618-8e18-78556a7f8989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:25.496995 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc0d49-04ef-4618-8e18-78556a7f8989 admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:25.497582 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cc0d49-04ef-4618-8e18-78556a7f8989 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:25.500429 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cc0d49-04ef-4618-8e18-78556a7f8989 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:25.531001 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 979/1982] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:57:25 2022] POST /identity/v3/users => generated 289 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:57:25.535133 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1048f24-262a-4542-b638-a574b790018a admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:25.545173 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1048f24-262a-4542-b638-a574b790018a admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:25.549966 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eef4b3-0640-439f-a1d1-9c49c51d6eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:25.550689 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eef4b3-0640-439f-a1d1-9c49c51d6eeb admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:25.550954 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eef4b3-0640-439f-a1d1-9c49c51d6eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:25.551307 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eef4b3-0640-439f-a1d1-9c49c51d6eeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:25.552041 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eef4b3-0640-439f-a1d1-9c49c51d6eeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:25.553665 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eef4b3-0640-439f-a1d1-9c49c51d6eeb admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:25.563789 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1004/1983] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:57:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 01:57:25.567861 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cc0d49-04ef-4618-8e18-78556a7f8989 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:25.579163 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cc0d49-04ef-4618-8e18-78556a7f8989 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:25.583766 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4250f500-3c99-4308-8f57-b26ab04ac17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:25.584437 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4250f500-3c99-4308-8f57-b26ab04ac17b admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:25.584716 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4250f500-3c99-4308-8f57-b26ab04ac17b admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:25.585132 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4250f500-3c99-4308-8f57-b26ab04ac17b admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:25.585891 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4250f500-3c99-4308-8f57-b26ab04ac17b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:25.587525 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4250f500-3c99-4308-8f57-b26ab04ac17b admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:25.597736 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 980/1984] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:57:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:57:25.602223 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eef4b3-0640-439f-a1d1-9c49c51d6eeb admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:25.612737 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eef4b3-0640-439f-a1d1-9c49c51d6eeb admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:25.618576 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8583e087-75d9-4201-ba7e-1b510ce79784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:25.619527 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583e087-75d9-4201-ba7e-1b510ce79784 admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:25.619790 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583e087-75d9-4201-ba7e-1b510ce79784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:25.620057 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583e087-75d9-4201-ba7e-1b510ce79784 admin admin] PATH_INFO: `/v3/projects/9951998e252845ca8927cf3432682075/users/f1e0e66beeab4be99ad4fd13a09afda6/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:25.635275 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8583e087-75d9-4201-ba7e-1b510ce79784 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9951998e252845ca8927cf3432682075, user_id=f1e0e66beeab4be99ad4fd13a09afda6, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:25.637882 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8583e087-75d9-4201-ba7e-1b510ce79784 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:25.653326 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1005/1985] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:57:25 2022] PUT /identity/v3/projects/9951998e252845ca8927cf3432682075/users/f1e0e66beeab4be99ad4fd13a09afda6/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:25.664116 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b0ca6-a134-4805-8e4e-eb74aed3223a None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:25.664406 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b0ca6-a134-4805-8e4e-eb74aed3223a None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:25.664683 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b0ca6-a134-4805-8e4e-eb74aed3223a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:25.693522 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-972b0ca6-a134-4805-8e4e-eb74aed3223a None None] MFA Rules not processed for user `f1e0e66beeab4be99ad4fd13a09afda6`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:25.694785 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-972b0ca6-a134-4805-8e4e-eb74aed3223a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:25.729027 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:25.766448 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 981/1986] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:25.771060 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8583e087-75d9-4201-ba7e-1b510ce79784 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:25.846121 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8583e087-75d9-4201-ba7e-1b510ce79784 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:25.852390 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cbfba8-4b0f-49c9-bfba-68ec0e8fb372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:25.853065 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbfba8-4b0f-49c9-bfba-68ec0e8fb372 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:25.853346 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbfba8-4b0f-49c9-bfba-68ec0e8fb372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:25.853634 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbfba8-4b0f-49c9-bfba-68ec0e8fb372 admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:25.854338 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cbfba8-4b0f-49c9-bfba-68ec0e8fb372 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:25.856334 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cbfba8-4b0f-49c9-bfba-68ec0e8fb372 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:25.884411 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1006/1987] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:57:25 2022] POST /identity/v3/projects => generated 375 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:57:25.888448 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972b0ca6-a134-4805-8e4e-eb74aed3223a None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:25.899738 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972b0ca6-a134-4805-8e4e-eb74aed3223a None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:25.957954 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb4afe5-a43f-4892-bf03-516b87e503d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:25.958668 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb4afe5-a43f-4892-bf03-516b87e503d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:25.958950 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb4afe5-a43f-4892-bf03-516b87e503d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:25.959253 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb4afe5-a43f-4892-bf03-516b87e503d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:25.959967 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb4afe5-a43f-4892-bf03-516b87e503d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:25.961893 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb4afe5-a43f-4892-bf03-516b87e503d0 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:25.985398 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 982/1988] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:57:25 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:57:25.986934 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cbfba8-4b0f-49c9-bfba-68ec0e8fb372 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:25.997824 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cbfba8-4b0f-49c9-bfba-68ec0e8fb372 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:26.002832 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c0fff-f011-407e-bc37-0dfe0b086b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:26.005118 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c0fff-f011-407e-bc37-0dfe0b086b04 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:26.005118 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c0fff-f011-407e-bc37-0dfe0b086b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:26.005118 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c0fff-f011-407e-bc37-0dfe0b086b04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:26.005118 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c0fff-f011-407e-bc37-0dfe0b086b04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:26.006829 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c0fff-f011-407e-bc37-0dfe0b086b04 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:26.021223 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1007/1989] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:57:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:57:26.026973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb4afe5-a43f-4892-bf03-516b87e503d0 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:26.038907 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb4afe5-a43f-4892-bf03-516b87e503d0 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:26.043768 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad31d85-b69a-4ec4-892e-b48a473bf3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:26.044694 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad31d85-b69a-4ec4-892e-b48a473bf3b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:26.044975 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad31d85-b69a-4ec4-892e-b48a473bf3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:26.045256 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad31d85-b69a-4ec4-892e-b48a473bf3b9 admin admin] PATH_INFO: `/v3/projects/5afdea1c69824282b3d48fd999d77c5c/users/0221b59c568c4c8aabb9a339df7f95d2/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:26.062812 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad31d85-b69a-4ec4-892e-b48a473bf3b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5afdea1c69824282b3d48fd999d77c5c, user_id=0221b59c568c4c8aabb9a339df7f95d2, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:26.065984 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad31d85-b69a-4ec4-892e-b48a473bf3b9 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:26.080606 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c0fff-f011-407e-bc37-0dfe0b086b04 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:26.082735 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-774c0fff-f011-407e-bc37-0dfe0b086b04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:26.084049 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 983/1990] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:57:26 2022] PUT /identity/v3/projects/5afdea1c69824282b3d48fd999d77c5c/users/0221b59c568c4c8aabb9a339df7f95d2/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:26.087603 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad31d85-b69a-4ec4-892e-b48a473bf3b9 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:26.103070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c0fff-f011-407e-bc37-0dfe0b086b04 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:26.198450 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad31d85-b69a-4ec4-892e-b48a473bf3b9 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:26.203936 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d85834e-aa1c-4c99-8e49-20eae1f3b959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:26.204757 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d85834e-aa1c-4c99-8e49-20eae1f3b959 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:26.205105 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d85834e-aa1c-4c99-8e49-20eae1f3b959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:26.205453 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d85834e-aa1c-4c99-8e49-20eae1f3b959 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:26.206264 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d85834e-aa1c-4c99-8e49-20eae1f3b959 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:26.208219 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d85834e-aa1c-4c99-8e49-20eae1f3b959 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:26.231328 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 984/1991] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:57:26 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 01:57:26.236376 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d85834e-aa1c-4c99-8e49-20eae1f3b959 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:26.250598 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d85834e-aa1c-4c99-8e49-20eae1f3b959 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:26.255932 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfce9ad-e852-40f6-952f-e4fc94fbfd51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:26.256243 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66fa200-0f43-4619-a9df-03b510fd6a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:26.257327 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fa200-0f43-4619-a9df-03b510fd6a82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:26.257718 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fa200-0f43-4619-a9df-03b510fd6a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:26.258106 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fa200-0f43-4619-a9df-03b510fd6a82 admin admin] PATH_INFO: `/v3/projects/5afdea1c69824282b3d48fd999d77c5c/users/0221b59c568c4c8aabb9a339df7f95d2/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:26.258805 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfce9ad-e852-40f6-952f-e4fc94fbfd51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:26.259104 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfce9ad-e852-40f6-952f-e4fc94fbfd51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:26.259418 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfce9ad-e852-40f6-952f-e4fc94fbfd51 admin admin] PATH_INFO: `/v3/users/aaad16e4b7f54ec88e5a6894f99e1b78` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:26.263082 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfce9ad-e852-40f6-952f-e4fc94fbfd51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aaad16e4b7f54ec88e5a6894f99e1b78)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:26.265384 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfce9ad-e852-40f6-952f-e4fc94fbfd51 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:26.266389 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66fa200-0f43-4619-a9df-03b510fd6a82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5afdea1c69824282b3d48fd999d77c5c, user_id=0221b59c568c4c8aabb9a339df7f95d2, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:26.270870 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66fa200-0f43-4619-a9df-03b510fd6a82 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:26.298311 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 985/1992] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:57:26 2022] PUT /identity/v3/projects/5afdea1c69824282b3d48fd999d77c5c/users/0221b59c568c4c8aabb9a339df7f95d2/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 01:57:26.313343 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:26.315420 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb50018-e29e-41e3-b9b0-38e182cb68af None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:26.316049 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb50018-e29e-41e3-b9b0-38e182cb68af None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:26.316049 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb50018-e29e-41e3-b9b0-38e182cb68af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:26.335250 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-4dfce9ad-e852-40f6-952f-e4fc94fbfd51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aaad16e4b7f54ec88e5a6894f99e1b78'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:26.351600 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-4dfce9ad-e852-40f6-952f-e4fc94fbfd51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aaad16e4b7f54ec88e5a6894f99e1b78'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:26.362691 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-4dfce9ad-e852-40f6-952f-e4fc94fbfd51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aaad16e4b7f54ec88e5a6894f99e1b78'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:26.373406 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-9bb50018-e29e-41e3-b9b0-38e182cb68af None None] MFA Rules not processed for user `0221b59c568c4c8aabb9a339df7f95d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:26.375759 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-9bb50018-e29e-41e3-b9b0-38e182cb68af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:26.379377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-4dfce9ad-e852-40f6-952f-e4fc94fbfd51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aaad16e4b7f54ec88e5a6894f99e1b78'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:26.381255 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-4dfce9ad-e852-40f6-952f-e4fc94fbfd51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aaad16e4b7f54ec88e5a6894f99e1b78'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:26.388544 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:26.399792 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1008/1993] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:57:26 2022] DELETE /identity/v3/users/aaad16e4b7f54ec88e5a6894f99e1b78 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 01:57:26.407616 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4035f4-2681-49d0-b154-e7e87a4ae071 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:26.407937 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4035f4-2681-49d0-b154-e7e87a4ae071 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:26.408225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4035f4-2681-49d0-b154-e7e87a4ae071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:26.442915 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-8a4035f4-2681-49d0-b154-e7e87a4ae071 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:26.446614 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-8a4035f4-2681-49d0-b154-e7e87a4ae071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:26.509943 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 986/1994] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:26 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 01:57:26.597802 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1009/1995] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:26.608024 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb50018-e29e-41e3-b9b0-38e182cb68af None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:26.613240 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4035f4-2681-49d0-b154-e7e87a4ae071 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:26.615266 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-9bb50018-e29e-41e3-b9b0-38e182cb68af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:26.615433 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-8a4035f4-2681-49d0-b154-e7e87a4ae071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:26.764447 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4035f4-2681-49d0-b154-e7e87a4ae071 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:26.773293 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb50018-e29e-41e3-b9b0-38e182cb68af None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:26.774605 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb726d8d-c779-4089-bba6-3179c3ba486d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:26.775270 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb726d8d-c779-4089-bba6-3179c3ba486d service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:26.775552 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb726d8d-c779-4089-bba6-3179c3ba486d service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:26.775823 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb726d8d-c779-4089-bba6-3179c3ba486d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:26.812455 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb726d8d-c779-4089-bba6-3179c3ba486d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:26.815081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb726d8d-c779-4089-bba6-3179c3ba486d service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:26.840228 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1010/1996] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:57:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:57:26.891198 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f791e51-4c99-4e08-b9dd-2577b0af3d80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:26.891901 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f791e51-4c99-4e08-b9dd-2577b0af3d80 service cinder] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:26.892182 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f791e51-4c99-4e08-b9dd-2577b0af3d80 service cinder] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:26.892688 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f791e51-4c99-4e08-b9dd-2577b0af3d80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:26.895858 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-3f791e51-4c99-4e08-b9dd-2577b0af3d80 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:26.965125 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f791e51-4c99-4e08-b9dd-2577b0af3d80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:26.967503 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f791e51-4c99-4e08-b9dd-2577b0af3d80 service cinder] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:26.980011 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 987/1997] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:57:26 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:57:28.305481 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb726d8d-c779-4089-bba6-3179c3ba486d service neutron] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:28.305864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-bb726d8d-c779-4089-bba6-3179c3ba486d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:28.331545 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb726d8d-c779-4089-bba6-3179c3ba486d service neutron] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:28.346197 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95da8eff-949e-4b87-b5ae-0064474158b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:28.359794 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da8eff-949e-4b87-b5ae-0064474158b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:28.360124 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da8eff-949e-4b87-b5ae-0064474158b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:28.360345 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da8eff-949e-4b87-b5ae-0064474158b5 admin admin] PATH_INFO: `/v3/projects/1fcb16e5774042bb869cdc085dd9b057` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:28.362353 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95da8eff-949e-4b87-b5ae-0064474158b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fcb16e5774042bb869cdc085dd9b057)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:28.369340 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95da8eff-949e-4b87-b5ae-0064474158b5 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:28.532890 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:28.539944 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-95da8eff-949e-4b87-b5ae-0064474158b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fcb16e5774042bb869cdc085dd9b057', 'request_id': 'req-95da8eff-949e-4b87-b5ae-0064474158b5'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:28.556442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-95da8eff-949e-4b87-b5ae-0064474158b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fcb16e5774042bb869cdc085dd9b057', 'request_id': 'req-95da8eff-949e-4b87-b5ae-0064474158b5'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:28.571289 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-95da8eff-949e-4b87-b5ae-0064474158b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fcb16e5774042bb869cdc085dd9b057', 'request_id': 'req-95da8eff-949e-4b87-b5ae-0064474158b5'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:28.606711 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-95da8eff-949e-4b87-b5ae-0064474158b5 admin admin] The token cache is being invalidate because project 1fcb16e5774042bb869cdc085dd9b057 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:57:28.607014 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-95da8eff-949e-4b87-b5ae-0064474158b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:28.612620 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-95da8eff-949e-4b87-b5ae-0064474158b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:28.614432 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1011/1998] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:57:28 2022] DELETE /identity/v3/projects/1fcb16e5774042bb869cdc085dd9b057 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:28.619134 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f791e51-4c99-4e08-b9dd-2577b0af3d80 service cinder] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:28.635179 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f791e51-4c99-4e08-b9dd-2577b0af3d80 service cinder] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:28.665628 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:28.743012 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347a43e-9d9d-4b20-b342-cf38e1660e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:28.745221 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347a43e-9d9d-4b20-b342-cf38e1660e11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:28.746752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347a43e-9d9d-4b20-b342-cf38e1660e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:28.748890 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347a43e-9d9d-4b20-b342-cf38e1660e11 admin admin] PATH_INFO: `/v3/users/b6ccae2e1c0c4545877ae09ada78829e` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:28.751347 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d347a43e-9d9d-4b20-b342-cf38e1660e11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6ccae2e1c0c4545877ae09ada78829e)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:28.755892 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d347a43e-9d9d-4b20-b342-cf38e1660e11 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:28.874404 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-d347a43e-9d9d-4b20-b342-cf38e1660e11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6ccae2e1c0c4545877ae09ada78829e'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:28.893114 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-d347a43e-9d9d-4b20-b342-cf38e1660e11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6ccae2e1c0c4545877ae09ada78829e'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:28.902533 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-d347a43e-9d9d-4b20-b342-cf38e1660e11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6ccae2e1c0c4545877ae09ada78829e'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:28.924956 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-d347a43e-9d9d-4b20-b342-cf38e1660e11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6ccae2e1c0c4545877ae09ada78829e'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:28.925942 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-d347a43e-9d9d-4b20-b342-cf38e1660e11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6ccae2e1c0c4545877ae09ada78829e'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:28.934045 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 988/1999] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:57:28 2022] DELETE /identity/v3/users/b6ccae2e1c0c4545877ae09ada78829e => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:29.547358 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95da8eff-949e-4b87-b5ae-0064474158b5 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:29.550547 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-95da8eff-949e-4b87-b5ae-0064474158b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:29.569167 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95da8eff-949e-4b87-b5ae-0064474158b5 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:29.661347 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e797d743-a15b-497f-91d6-9c7f29d8c1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:29.662220 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797d743-a15b-497f-91d6-9c7f29d8c1fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:29.662496 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797d743-a15b-497f-91d6-9c7f29d8c1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:29.662767 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797d743-a15b-497f-91d6-9c7f29d8c1fe admin admin] PATH_INFO: `/v3/projects/7c8e7eaa8a624912b74063acb9dc909a` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:29.664307 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e797d743-a15b-497f-91d6-9c7f29d8c1fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c8e7eaa8a624912b74063acb9dc909a)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:29.666266 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e797d743-a15b-497f-91d6-9c7f29d8c1fe admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:29.731058 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-e797d743-a15b-497f-91d6-9c7f29d8c1fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c8e7eaa8a624912b74063acb9dc909a', 'request_id': 'req-e797d743-a15b-497f-91d6-9c7f29d8c1fe'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:29.739354 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-e797d743-a15b-497f-91d6-9c7f29d8c1fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c8e7eaa8a624912b74063acb9dc909a', 'request_id': 'req-e797d743-a15b-497f-91d6-9c7f29d8c1fe'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:29.748700 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-e797d743-a15b-497f-91d6-9c7f29d8c1fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c8e7eaa8a624912b74063acb9dc909a', 'request_id': 'req-e797d743-a15b-497f-91d6-9c7f29d8c1fe'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:29.764186 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-e797d743-a15b-497f-91d6-9c7f29d8c1fe admin admin] The token cache is being invalidate because project 7c8e7eaa8a624912b74063acb9dc909a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:57:29.764712 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-e797d743-a15b-497f-91d6-9c7f29d8c1fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:29.765393 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-e797d743-a15b-497f-91d6-9c7f29d8c1fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:29.767255 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1012/2000] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:57:29 2022] DELETE /identity/v3/projects/7c8e7eaa8a624912b74063acb9dc909a => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:29.772252 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b32bb6-da9c-4658-a809-e37e50f143de None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:29.773754 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b32bb6-da9c-4658-a809-e37e50f143de None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:29.774303 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b32bb6-da9c-4658-a809-e37e50f143de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:29.799454 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-57b32bb6-da9c-4658-a809-e37e50f143de None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:29.800752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-57b32bb6-da9c-4658-a809-e37e50f143de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:29.864302 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 989/2001] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:29.872103 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834444f-d086-4d01-9c68-e2a514edb6a5 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:29.872392 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834444f-d086-4d01-9c68-e2a514edb6a5 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:29.872727 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834444f-d086-4d01-9c68-e2a514edb6a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:29.892039 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-2834444f-d086-4d01-9c68-e2a514edb6a5 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:29.893254 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-2834444f-d086-4d01-9c68-e2a514edb6a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:29.901689 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1013/2002] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:29.906042 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b32bb6-da9c-4658-a809-e37e50f143de None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:29.916948 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b32bb6-da9c-4658-a809-e37e50f143de None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:29.921762 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3256e0e8-bd03-495c-a6df-0d8b04c7a6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:29.922427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256e0e8-bd03-495c-a6df-0d8b04c7a6fe admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:29.922707 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256e0e8-bd03-495c-a6df-0d8b04c7a6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:29.923103 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256e0e8-bd03-495c-a6df-0d8b04c7a6fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:29.924196 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3256e0e8-bd03-495c-a6df-0d8b04c7a6fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:57:29.924500 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3256e0e8-bd03-495c-a6df-0d8b04c7a6fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:29.926425 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3256e0e8-bd03-495c-a6df-0d8b04c7a6fe admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:29.940136 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 990/2003] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 01:57:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:57:29.944417 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834444f-d086-4d01-9c68-e2a514edb6a5 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:29.954372 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834444f-d086-4d01-9c68-e2a514edb6a5 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:29.959549 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdffdd3-1408-49d9-b9e8-4aa8cb73d350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:29.960368 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdffdd3-1408-49d9-b9e8-4aa8cb73d350 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:29.960953 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdffdd3-1408-49d9-b9e8-4aa8cb73d350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:29.966783 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdffdd3-1408-49d9-b9e8-4aa8cb73d350 admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:29.967642 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdffdd3-1408-49d9-b9e8-4aa8cb73d350 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:29.969836 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdffdd3-1408-49d9-b9e8-4aa8cb73d350 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:30.005862 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1014/2004] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:57:29 2022] POST /identity/v3/projects => generated 387 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:57:30.007356 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3256e0e8-bd03-495c-a6df-0d8b04c7a6fe admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:30.017722 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3256e0e8-bd03-495c-a6df-0d8b04c7a6fe admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:30.040569 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:30.083479 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d0b50a-0f66-44de-a039-9e86e0afceb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:30.084191 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0b50a-0f66-44de-a039-9e86e0afceb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:30.084450 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0b50a-0f66-44de-a039-9e86e0afceb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:30.084733 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0b50a-0f66-44de-a039-9e86e0afceb6 admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:30.085443 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d0b50a-0f66-44de-a039-9e86e0afceb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:30.087392 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d0b50a-0f66-44de-a039-9e86e0afceb6 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:30.111204 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 991/2005] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:57:30 2022] POST /identity/v3/users => generated 294 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 01:57:30.112508 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdffdd3-1408-49d9-b9e8-4aa8cb73d350 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:30.123051 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdffdd3-1408-49d9-b9e8-4aa8cb73d350 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:30.132435 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fc1d48-6a21-4123-8230-74d0983d6163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:30.133180 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fc1d48-6a21-4123-8230-74d0983d6163 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:30.133528 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fc1d48-6a21-4123-8230-74d0983d6163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:30.133865 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fc1d48-6a21-4123-8230-74d0983d6163 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:30.134675 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fc1d48-6a21-4123-8230-74d0983d6163 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:30.136313 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fc1d48-6a21-4123-8230-74d0983d6163 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:30.147270 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1015/2006] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:57:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:57:30.151306 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d0b50a-0f66-44de-a039-9e86e0afceb6 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:30.161305 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d0b50a-0f66-44de-a039-9e86e0afceb6 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:30.165904 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a2c52-c524-4ead-8563-970ec4431e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:30.166579 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a2c52-c524-4ead-8563-970ec4431e6a admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:30.166864 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a2c52-c524-4ead-8563-970ec4431e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:30.167143 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a2c52-c524-4ead-8563-970ec4431e6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:30.167944 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79a2c52-c524-4ead-8563-970ec4431e6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:30.169520 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79a2c52-c524-4ead-8563-970ec4431e6a admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:30.183733 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 992/2007] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:57:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:57:30.184791 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fc1d48-6a21-4123-8230-74d0983d6163 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:30.195299 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fc1d48-6a21-4123-8230-74d0983d6163 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:30.200196 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a019471b-2e06-4717-bd7e-ae0099b4abd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:30.201163 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019471b-2e06-4717-bd7e-ae0099b4abd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:30.201523 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019471b-2e06-4717-bd7e-ae0099b4abd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:30.201857 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019471b-2e06-4717-bd7e-ae0099b4abd6 admin admin] PATH_INFO: `/v3/projects/8dc01fc82ebe443aa47441c47ffb1506/users/6cb587c9e3e447bcbbb59097fce7e13a/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:30.218504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a019471b-2e06-4717-bd7e-ae0099b4abd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dc01fc82ebe443aa47441c47ffb1506, user_id=6cb587c9e3e447bcbbb59097fce7e13a, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:30.221320 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a019471b-2e06-4717-bd7e-ae0099b4abd6 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:30.233826 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:30.238963 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1016/2008] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:57:30 2022] PUT /identity/v3/projects/8dc01fc82ebe443aa47441c47ffb1506/users/6cb587c9e3e447bcbbb59097fce7e13a/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 01:57:30.249803 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2d378-2f65-4385-92ce-63607fdad0ea None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:30.250097 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2d378-2f65-4385-92ce-63607fdad0ea None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:30.250466 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2d378-2f65-4385-92ce-63607fdad0ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:30.280209 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-83e2d378-2f65-4385-92ce-63607fdad0ea None None] MFA Rules not processed for user `6cb587c9e3e447bcbbb59097fce7e13a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:30.281629 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-83e2d378-2f65-4385-92ce-63607fdad0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:30.348474 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 993/2009] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:30.353167 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a019471b-2e06-4717-bd7e-ae0099b4abd6 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:30.421370 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a019471b-2e06-4717-bd7e-ae0099b4abd6 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:30.426421 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8d6b15-d365-4ecc-b62d-ab8c5099b5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:30.427086 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d6b15-d365-4ecc-b62d-ab8c5099b5eb admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:30.427396 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d6b15-d365-4ecc-b62d-ab8c5099b5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:30.427669 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d6b15-d365-4ecc-b62d-ab8c5099b5eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:30.428370 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8d6b15-d365-4ecc-b62d-ab8c5099b5eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:30.430299 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8d6b15-d365-4ecc-b62d-ab8c5099b5eb admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:30.462005 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1017/2010] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:57:30 2022] POST /identity/v3/projects => generated 389 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:57:30.465998 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2d378-2f65-4385-92ce-63607fdad0ea None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:30.482056 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2d378-2f65-4385-92ce-63607fdad0ea None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:30.551250 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61779429-b4aa-469a-986e-909d98c69d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:30.551945 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61779429-b4aa-469a-986e-909d98c69d93 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:30.552254 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61779429-b4aa-469a-986e-909d98c69d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:30.552656 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61779429-b4aa-469a-986e-909d98c69d93 admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:30.553653 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61779429-b4aa-469a-986e-909d98c69d93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:30.556675 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61779429-b4aa-469a-986e-909d98c69d93 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:30.578207 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 994/2011] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:57:30 2022] POST /identity/v3/users => generated 301 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:57:30.581950 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8d6b15-d365-4ecc-b62d-ab8c5099b5eb admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:30.592869 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8d6b15-d365-4ecc-b62d-ab8c5099b5eb admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:30.597581 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d2505-9228-42d2-86f9-0a7a9fe5f736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:30.598376 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d2505-9228-42d2-86f9-0a7a9fe5f736 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:30.598661 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d2505-9228-42d2-86f9-0a7a9fe5f736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:30.598934 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d2505-9228-42d2-86f9-0a7a9fe5f736 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:30.599705 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31d2505-9228-42d2-86f9-0a7a9fe5f736 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:30.601369 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31d2505-9228-42d2-86f9-0a7a9fe5f736 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:30.612813 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1018/2012] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:57:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:57:30.616975 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61779429-b4aa-469a-986e-909d98c69d93 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:30.628915 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61779429-b4aa-469a-986e-909d98c69d93 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:30.633835 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5ed7a2-a090-4998-aab1-4562e1ff56ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:30.634736 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ed7a2-a090-4998-aab1-4562e1ff56ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:30.635016 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ed7a2-a090-4998-aab1-4562e1ff56ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:30.635335 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ed7a2-a090-4998-aab1-4562e1ff56ba admin admin] PATH_INFO: `/v3/projects/da05af85d43c4b68956ef9d02b5f7bfe/users/9558c45f20134136a99fea5b2912763e/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:30.652625 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5ed7a2-a090-4998-aab1-4562e1ff56ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=da05af85d43c4b68956ef9d02b5f7bfe, user_id=9558c45f20134136a99fea5b2912763e, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:30.655546 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5ed7a2-a090-4998-aab1-4562e1ff56ba admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:30.665126 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:30.681937 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 995/2013] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:57:30 2022] PUT /identity/v3/projects/da05af85d43c4b68956ef9d02b5f7bfe/users/9558c45f20134136a99fea5b2912763e/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:30.686298 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d2505-9228-42d2-86f9-0a7a9fe5f736 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:30.699988 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d2505-9228-42d2-86f9-0a7a9fe5f736 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:30.759557 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3753bb49-8ea2-45c9-903f-acf3f840fc66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:30.760206 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753bb49-8ea2-45c9-903f-acf3f840fc66 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:30.760489 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753bb49-8ea2-45c9-903f-acf3f840fc66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:30.760763 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753bb49-8ea2-45c9-903f-acf3f840fc66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:30.761499 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3753bb49-8ea2-45c9-903f-acf3f840fc66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:30.763128 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3753bb49-8ea2-45c9-903f-acf3f840fc66 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:30.773885 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1019/2014] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:57:30 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 01:57:30.778205 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5ed7a2-a090-4998-aab1-4562e1ff56ba admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:30.789072 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5ed7a2-a090-4998-aab1-4562e1ff56ba admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:30.794026 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e8ea4-f6eb-4883-8a36-cf2f00876215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:30.795122 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e8ea4-f6eb-4883-8a36-cf2f00876215 admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:30.795451 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e8ea4-f6eb-4883-8a36-cf2f00876215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:30.795728 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e8ea4-f6eb-4883-8a36-cf2f00876215 admin admin] PATH_INFO: `/v3/projects/da05af85d43c4b68956ef9d02b5f7bfe/users/9558c45f20134136a99fea5b2912763e/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:30.797791 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43e8ea4-f6eb-4883-8a36-cf2f00876215 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da05af85d43c4b68956ef9d02b5f7bfe, user_id=9558c45f20134136a99fea5b2912763e, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:30.800942 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43e8ea4-f6eb-4883-8a36-cf2f00876215 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:30.812115 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 996/2015] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:57:30 2022] PUT /identity/v3/projects/da05af85d43c4b68956ef9d02b5f7bfe/users/9558c45f20134136a99fea5b2912763e/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:30.824117 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8c60f-b692-4826-b3d7-e2d41a31597f None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:30.824431 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8c60f-b692-4826-b3d7-e2d41a31597f None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:30.824711 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8c60f-b692-4826-b3d7-e2d41a31597f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:30.851488 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-5fc8c60f-b692-4826-b3d7-e2d41a31597f None None] MFA Rules not processed for user `9558c45f20134136a99fea5b2912763e`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:30.852740 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-5fc8c60f-b692-4826-b3d7-e2d41a31597f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:30.914492 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:30.924586 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1020/2016] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:30 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:30.937609 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e8ea4-f6eb-4883-8a36-cf2f00876215 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:30.939482 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-d43e8ea4-f6eb-4883-8a36-cf2f00876215 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:30.998530 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e8ea4-f6eb-4883-8a36-cf2f00876215 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:31.002274 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b90e897-8fef-4efd-8588-b0aa3d280e62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:31.002926 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90e897-8fef-4efd-8588-b0aa3d280e62 service cinder] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:31.003205 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90e897-8fef-4efd-8588-b0aa3d280e62 service cinder] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:31.003504 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b90e897-8fef-4efd-8588-b0aa3d280e62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:31.012827 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b90e897-8fef-4efd-8588-b0aa3d280e62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:31.014980 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b90e897-8fef-4efd-8588-b0aa3d280e62 service cinder] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:31.026614 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 997/2017] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:57:30 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:57:31.231751 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc8c60f-b692-4826-b3d7-e2d41a31597f None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:31.300905 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc8c60f-b692-4826-b3d7-e2d41a31597f None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:31.306570 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea0daad-6e91-46be-8955-e986a21f5bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:31.307566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0daad-6e91-46be-8955-e986a21f5bd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:31.307841 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0daad-6e91-46be-8955-e986a21f5bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:31.308110 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0daad-6e91-46be-8955-e986a21f5bd0 admin admin] PATH_INFO: `/v3/users/6cb587c9e3e447bcbbb59097fce7e13a` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:31.309442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea0daad-6e91-46be-8955-e986a21f5bd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cb587c9e3e447bcbbb59097fce7e13a)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:31.311455 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea0daad-6e91-46be-8955-e986a21f5bd0 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:31.361198 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-aea0daad-6e91-46be-8955-e986a21f5bd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cb587c9e3e447bcbbb59097fce7e13a'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:31.373430 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-aea0daad-6e91-46be-8955-e986a21f5bd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cb587c9e3e447bcbbb59097fce7e13a'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:31.382443 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-aea0daad-6e91-46be-8955-e986a21f5bd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cb587c9e3e447bcbbb59097fce7e13a'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:31.389513 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-aea0daad-6e91-46be-8955-e986a21f5bd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cb587c9e3e447bcbbb59097fce7e13a'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:31.390152 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-aea0daad-6e91-46be-8955-e986a21f5bd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cb587c9e3e447bcbbb59097fce7e13a'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:31.396171 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1021/2018] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:57:31 2022] DELETE /identity/v3/users/6cb587c9e3e447bcbbb59097fce7e13a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:31.400745 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136c52b-6826-4fa1-b319-056d2af6a703 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:31.401035 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136c52b-6826-4fa1-b319-056d2af6a703 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:31.401313 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136c52b-6826-4fa1-b319-056d2af6a703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:31.419654 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-f136c52b-6826-4fa1-b319-056d2af6a703 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:31.420857 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-f136c52b-6826-4fa1-b319-056d2af6a703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:31.488097 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 998/2019] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:31.497884 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea0daad-6e91-46be-8955-e986a21f5bd0 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:31.499357 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-aea0daad-6e91-46be-8955-e986a21f5bd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:31.557710 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea0daad-6e91-46be-8955-e986a21f5bd0 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:31.561449 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a519ea-acaf-41b2-a6ec-24720016f9bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:31.562090 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a519ea-acaf-41b2-a6ec-24720016f9bb service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:31.562365 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a519ea-acaf-41b2-a6ec-24720016f9bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:31.562632 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a519ea-acaf-41b2-a6ec-24720016f9bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:31.572463 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a519ea-acaf-41b2-a6ec-24720016f9bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:31.573947 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a519ea-acaf-41b2-a6ec-24720016f9bb service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:31.583768 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1022/2020] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:57:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:57:31.856973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f136c52b-6826-4fa1-b319-056d2af6a703 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:31.858515 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-f136c52b-6826-4fa1-b319-056d2af6a703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:31.871458 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f136c52b-6826-4fa1-b319-056d2af6a703 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:31.877452 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb07cf-e86d-4409-829a-4bfd74268e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:31.878471 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb07cf-e86d-4409-829a-4bfd74268e8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:31.879457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb07cf-e86d-4409-829a-4bfd74268e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:31.880108 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb07cf-e86d-4409-829a-4bfd74268e8c admin admin] PATH_INFO: `/v3/projects/8dc01fc82ebe443aa47441c47ffb1506` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:31.881564 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bb07cf-e86d-4409-829a-4bfd74268e8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dc01fc82ebe443aa47441c47ffb1506)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:31.883522 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bb07cf-e86d-4409-829a-4bfd74268e8c admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:31.932583 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-b9bb07cf-e86d-4409-829a-4bfd74268e8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dc01fc82ebe443aa47441c47ffb1506', 'request_id': 'req-b9bb07cf-e86d-4409-829a-4bfd74268e8c'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:31.938698 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-b9bb07cf-e86d-4409-829a-4bfd74268e8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dc01fc82ebe443aa47441c47ffb1506', 'request_id': 'req-b9bb07cf-e86d-4409-829a-4bfd74268e8c'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:31.947410 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-b9bb07cf-e86d-4409-829a-4bfd74268e8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dc01fc82ebe443aa47441c47ffb1506', 'request_id': 'req-b9bb07cf-e86d-4409-829a-4bfd74268e8c'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:31.959611 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-b9bb07cf-e86d-4409-829a-4bfd74268e8c admin admin] The token cache is being invalidate because project 8dc01fc82ebe443aa47441c47ffb1506 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:57:31.960011 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-b9bb07cf-e86d-4409-829a-4bfd74268e8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:31.961082 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-b9bb07cf-e86d-4409-829a-4bfd74268e8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:31.963649 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 999/2021] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:57:31 2022] DELETE /identity/v3/projects/8dc01fc82ebe443aa47441c47ffb1506 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:31.967956 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a519ea-acaf-41b2-a6ec-24720016f9bb service neutron] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:31.978046 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a519ea-acaf-41b2-a6ec-24720016f9bb service neutron] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:31.995876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:32.046819 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef05405-12c7-4204-8ed1-b74312a285d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:32.047730 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef05405-12c7-4204-8ed1-b74312a285d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:32.048004 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef05405-12c7-4204-8ed1-b74312a285d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:32.048276 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef05405-12c7-4204-8ed1-b74312a285d3 admin admin] PATH_INFO: `/v3/users/9558c45f20134136a99fea5b2912763e` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:32.050533 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef05405-12c7-4204-8ed1-b74312a285d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9558c45f20134136a99fea5b2912763e)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:32.052748 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef05405-12c7-4204-8ed1-b74312a285d3 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:32.110477 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-5ef05405-12c7-4204-8ed1-b74312a285d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9558c45f20134136a99fea5b2912763e'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:32.125661 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-5ef05405-12c7-4204-8ed1-b74312a285d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9558c45f20134136a99fea5b2912763e'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:32.137835 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-5ef05405-12c7-4204-8ed1-b74312a285d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9558c45f20134136a99fea5b2912763e'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:32.152019 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-5ef05405-12c7-4204-8ed1-b74312a285d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9558c45f20134136a99fea5b2912763e'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:32.153552 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-5ef05405-12c7-4204-8ed1-b74312a285d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9558c45f20134136a99fea5b2912763e'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:32.161566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1023/2022] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:57:31 2022] DELETE /identity/v3/users/9558c45f20134136a99fea5b2912763e => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:32.667701 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb07cf-e86d-4409-829a-4bfd74268e8c admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:32.669248 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-b9bb07cf-e86d-4409-829a-4bfd74268e8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:32.682914 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb07cf-e86d-4409-829a-4bfd74268e8c admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:32.691258 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:32.747618 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2da197-3604-4104-a9b5-eba5afa29138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:32.748507 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2da197-3604-4104-a9b5-eba5afa29138 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:32.748790 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2da197-3604-4104-a9b5-eba5afa29138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:32.749190 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2da197-3604-4104-a9b5-eba5afa29138 admin admin] PATH_INFO: `/v3/projects/da05af85d43c4b68956ef9d02b5f7bfe` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:32.750435 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2da197-3604-4104-a9b5-eba5afa29138 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da05af85d43c4b68956ef9d02b5f7bfe)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:32.752395 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2da197-3604-4104-a9b5-eba5afa29138 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:32.811914 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-ab2da197-3604-4104-a9b5-eba5afa29138 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da05af85d43c4b68956ef9d02b5f7bfe', 'request_id': 'req-ab2da197-3604-4104-a9b5-eba5afa29138'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:32.818170 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-ab2da197-3604-4104-a9b5-eba5afa29138 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da05af85d43c4b68956ef9d02b5f7bfe', 'request_id': 'req-ab2da197-3604-4104-a9b5-eba5afa29138'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:32.825635 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-ab2da197-3604-4104-a9b5-eba5afa29138 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da05af85d43c4b68956ef9d02b5f7bfe', 'request_id': 'req-ab2da197-3604-4104-a9b5-eba5afa29138'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:32.836322 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-ab2da197-3604-4104-a9b5-eba5afa29138 admin admin] The token cache is being invalidate because project da05af85d43c4b68956ef9d02b5f7bfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:57:32.836514 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-ab2da197-3604-4104-a9b5-eba5afa29138 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:32.837143 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-ab2da197-3604-4104-a9b5-eba5afa29138 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:32.838930 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1000/2023] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:57:32 2022] DELETE /identity/v3/projects/da05af85d43c4b68956ef9d02b5f7bfe => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:33.256029 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef05405-12c7-4204-8ed1-b74312a285d3 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:33.258808 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-5ef05405-12c7-4204-8ed1-b74312a285d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:33.321103 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef05405-12c7-4204-8ed1-b74312a285d3 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:33.369908 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e210b-0cf3-4578-bb10-d7366119b46b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:33.370567 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e210b-0cf3-4578-bb10-d7366119b46b service cinder] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:33.370841 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e210b-0cf3-4578-bb10-d7366119b46b service cinder] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:33.371108 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e210b-0cf3-4578-bb10-d7366119b46b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:33.373086 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-dc7e210b-0cf3-4578-bb10-d7366119b46b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:33.379658 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:33.441725 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7e210b-0cf3-4578-bb10-d7366119b46b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:33.443178 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7e210b-0cf3-4578-bb10-d7366119b46b service cinder] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:33.453171 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1024/2024] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:57:33 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:57:40.524283 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2da197-3604-4104-a9b5-eba5afa29138 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:40.525821 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-ab2da197-3604-4104-a9b5-eba5afa29138 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:40.633562 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2da197-3604-4104-a9b5-eba5afa29138 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:40.638372 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0411343c-41d0-46ac-aaed-4a7f4add21cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:40.639266 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411343c-41d0-46ac-aaed-4a7f4add21cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:40.639593 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411343c-41d0-46ac-aaed-4a7f4add21cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:40.639880 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411343c-41d0-46ac-aaed-4a7f4add21cf admin admin] PATH_INFO: `/v3/users/5d066742ffbc4c78b40252f6dabe6efb` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:40.641093 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0411343c-41d0-46ac-aaed-4a7f4add21cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d066742ffbc4c78b40252f6dabe6efb)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:40.643027 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0411343c-41d0-46ac-aaed-4a7f4add21cf admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:40.691391 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:40.694872 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-0411343c-41d0-46ac-aaed-4a7f4add21cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d066742ffbc4c78b40252f6dabe6efb'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:40.709068 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-0411343c-41d0-46ac-aaed-4a7f4add21cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d066742ffbc4c78b40252f6dabe6efb'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:40.717423 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-0411343c-41d0-46ac-aaed-4a7f4add21cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d066742ffbc4c78b40252f6dabe6efb'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:40.724522 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-0411343c-41d0-46ac-aaed-4a7f4add21cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d066742ffbc4c78b40252f6dabe6efb'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:40.725148 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-0411343c-41d0-46ac-aaed-4a7f4add21cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d066742ffbc4c78b40252f6dabe6efb'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:40.730989 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1001/2025] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:57:40 2022] DELETE /identity/v3/users/5d066742ffbc4c78b40252f6dabe6efb => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:40.735785 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87344669-fc4a-4784-9edf-1d4a069141c7 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:40.736279 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87344669-fc4a-4784-9edf-1d4a069141c7 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:40.736414 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87344669-fc4a-4784-9edf-1d4a069141c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:40.754160 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-87344669-fc4a-4784-9edf-1d4a069141c7 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:40.755532 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-87344669-fc4a-4784-9edf-1d4a069141c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:40.823101 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1025/2026] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:40.832962 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0411343c-41d0-46ac-aaed-4a7f4add21cf admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:40.834623 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-0411343c-41d0-46ac-aaed-4a7f4add21cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:40.894677 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0411343c-41d0-46ac-aaed-4a7f4add21cf admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:40.901101 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5977fa-9e00-465c-b7e9-79754f378a1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:40.902032 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5977fa-9e00-465c-b7e9-79754f378a1c service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:40.902406 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5977fa-9e00-465c-b7e9-79754f378a1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:40.902759 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5977fa-9e00-465c-b7e9-79754f378a1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:40.915561 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5977fa-9e00-465c-b7e9-79754f378a1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:40.917003 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5977fa-9e00-465c-b7e9-79754f378a1c service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:40.927250 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1002/2027] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:57:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:57:41.202104 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87344669-fc4a-4784-9edf-1d4a069141c7 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:41.203817 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-87344669-fc4a-4784-9edf-1d4a069141c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:41.221314 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87344669-fc4a-4784-9edf-1d4a069141c7 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:41.228065 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69921c24-8770-4193-897f-c8f9b6ae3f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:41.228959 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69921c24-8770-4193-897f-c8f9b6ae3f72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:41.229372 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69921c24-8770-4193-897f-c8f9b6ae3f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:41.229678 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69921c24-8770-4193-897f-c8f9b6ae3f72 admin admin] PATH_INFO: `/v3/projects/2f22a5c2b7dd4b4a9b1487dbff55eace` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:41.231009 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69921c24-8770-4193-897f-c8f9b6ae3f72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f22a5c2b7dd4b4a9b1487dbff55eace)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:41.232998 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69921c24-8770-4193-897f-c8f9b6ae3f72 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:41.284394 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-69921c24-8770-4193-897f-c8f9b6ae3f72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f22a5c2b7dd4b4a9b1487dbff55eace', 'request_id': 'req-69921c24-8770-4193-897f-c8f9b6ae3f72'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:41.290784 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-69921c24-8770-4193-897f-c8f9b6ae3f72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f22a5c2b7dd4b4a9b1487dbff55eace', 'request_id': 'req-69921c24-8770-4193-897f-c8f9b6ae3f72'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:41.301371 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-69921c24-8770-4193-897f-c8f9b6ae3f72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f22a5c2b7dd4b4a9b1487dbff55eace', 'request_id': 'req-69921c24-8770-4193-897f-c8f9b6ae3f72'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:41.312729 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-69921c24-8770-4193-897f-c8f9b6ae3f72 admin admin] The token cache is being invalidate because project 2f22a5c2b7dd4b4a9b1487dbff55eace was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:57:41.313015 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-69921c24-8770-4193-897f-c8f9b6ae3f72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:41.313634 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-69921c24-8770-4193-897f-c8f9b6ae3f72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:41.315174 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1026/2028] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:57:41 2022] DELETE /identity/v3/projects/2f22a5c2b7dd4b4a9b1487dbff55eace => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:41.319187 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5977fa-9e00-465c-b7e9-79754f378a1c service neutron] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:41.329036 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5977fa-9e00-465c-b7e9-79754f378a1c service neutron] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:41.391285 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357cb860-fb2d-4cbc-8cdd-89bf70d080ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:41.392157 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357cb860-fb2d-4cbc-8cdd-89bf70d080ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:41.392436 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357cb860-fb2d-4cbc-8cdd-89bf70d080ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:41.392707 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357cb860-fb2d-4cbc-8cdd-89bf70d080ae admin admin] PATH_INFO: `/v3/users/52068a13e7db4cf0a2e1f0f4b456bd23` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:41.394080 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357cb860-fb2d-4cbc-8cdd-89bf70d080ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=52068a13e7db4cf0a2e1f0f4b456bd23)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:41.396172 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357cb860-fb2d-4cbc-8cdd-89bf70d080ae admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:41.449222 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-357cb860-fb2d-4cbc-8cdd-89bf70d080ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52068a13e7db4cf0a2e1f0f4b456bd23'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:41.461440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-357cb860-fb2d-4cbc-8cdd-89bf70d080ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52068a13e7db4cf0a2e1f0f4b456bd23'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:41.463103 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:41.471242 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-357cb860-fb2d-4cbc-8cdd-89bf70d080ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52068a13e7db4cf0a2e1f0f4b456bd23'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:41.479643 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-357cb860-fb2d-4cbc-8cdd-89bf70d080ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52068a13e7db4cf0a2e1f0f4b456bd23'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:41.480298 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-357cb860-fb2d-4cbc-8cdd-89bf70d080ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52068a13e7db4cf0a2e1f0f4b456bd23'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:41.486434 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1003/2029] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:57:41 2022] DELETE /identity/v3/users/52068a13e7db4cf0a2e1f0f4b456bd23 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 01:57:41.767172 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69921c24-8770-4193-897f-c8f9b6ae3f72 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:41.768834 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-69921c24-8770-4193-897f-c8f9b6ae3f72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:41.782930 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69921c24-8770-4193-897f-c8f9b6ae3f72 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:41.809784 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:41.843284 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b46afa6-efbd-4ea4-a836-36583315cb5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:41.844110 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b46afa6-efbd-4ea4-a836-36583315cb5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:41.844384 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b46afa6-efbd-4ea4-a836-36583315cb5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:41.844669 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b46afa6-efbd-4ea4-a836-36583315cb5e admin admin] PATH_INFO: `/v3/projects/58941b5e74fe431687a65fa7c3b77e23` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:41.845854 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b46afa6-efbd-4ea4-a836-36583315cb5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=58941b5e74fe431687a65fa7c3b77e23)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:41.847798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b46afa6-efbd-4ea4-a836-36583315cb5e admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:41.894608 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-0b46afa6-efbd-4ea4-a836-36583315cb5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58941b5e74fe431687a65fa7c3b77e23', 'request_id': 'req-0b46afa6-efbd-4ea4-a836-36583315cb5e'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:41.900723 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-0b46afa6-efbd-4ea4-a836-36583315cb5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58941b5e74fe431687a65fa7c3b77e23', 'request_id': 'req-0b46afa6-efbd-4ea4-a836-36583315cb5e'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:41.910731 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-0b46afa6-efbd-4ea4-a836-36583315cb5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58941b5e74fe431687a65fa7c3b77e23', 'request_id': 'req-0b46afa6-efbd-4ea4-a836-36583315cb5e'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:41.927844 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-0b46afa6-efbd-4ea4-a836-36583315cb5e admin admin] The token cache is being invalidate because project 58941b5e74fe431687a65fa7c3b77e23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:57:41.928229 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-0b46afa6-efbd-4ea4-a836-36583315cb5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:41.929311 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-0b46afa6-efbd-4ea4-a836-36583315cb5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:41.931981 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1027/2030] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:57:41 2022] DELETE /identity/v3/projects/58941b5e74fe431687a65fa7c3b77e23 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:41.938343 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f72dd7-8fa8-43a6-bd95-0daea87b8b3f None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:41.938837 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f72dd7-8fa8-43a6-bd95-0daea87b8b3f None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:41.938929 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f72dd7-8fa8-43a6-bd95-0daea87b8b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:41.958006 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-e5f72dd7-8fa8-43a6-bd95-0daea87b8b3f None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:41.959235 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-e5f72dd7-8fa8-43a6-bd95-0daea87b8b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:42.022519 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1004/2031] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:42.031475 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb43b08-dd8c-47df-8c9a-86929ae6d2ef None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:42.031763 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb43b08-dd8c-47df-8c9a-86929ae6d2ef None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:42.032039 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb43b08-dd8c-47df-8c9a-86929ae6d2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:42.051570 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-dcb43b08-dd8c-47df-8c9a-86929ae6d2ef None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:42.052794 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-dcb43b08-dd8c-47df-8c9a-86929ae6d2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:42.061000 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1028/2032] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:42.065526 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f72dd7-8fa8-43a6-bd95-0daea87b8b3f None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:42.075445 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f72dd7-8fa8-43a6-bd95-0daea87b8b3f None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:42.080307 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69247f9e-ac24-40a2-b94b-82e6780a301e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:42.080962 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69247f9e-ac24-40a2-b94b-82e6780a301e admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:42.081246 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69247f9e-ac24-40a2-b94b-82e6780a301e admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:42.081533 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69247f9e-ac24-40a2-b94b-82e6780a301e admin admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:42.082264 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69247f9e-ac24-40a2-b94b-82e6780a301e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:57:42.082666 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69247f9e-ac24-40a2-b94b-82e6780a301e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:42.084322 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69247f9e-ac24-40a2-b94b-82e6780a301e admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:42.096698 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1005/2033] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 01:57:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:57:42.101090 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb43b08-dd8c-47df-8c9a-86929ae6d2ef None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:42.110875 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb43b08-dd8c-47df-8c9a-86929ae6d2ef None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:42.116053 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aa8cdc-4de7-491a-b0b5-5c11b4f0fd53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:42.116695 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa8cdc-4de7-491a-b0b5-5c11b4f0fd53 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:42.116975 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa8cdc-4de7-491a-b0b5-5c11b4f0fd53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:42.117250 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa8cdc-4de7-491a-b0b5-5c11b4f0fd53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:42.117947 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aa8cdc-4de7-491a-b0b5-5c11b4f0fd53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:42.120051 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aa8cdc-4de7-491a-b0b5-5c11b4f0fd53 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:42.147305 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1029/2034] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:57:42 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:57:42.151474 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69247f9e-ac24-40a2-b94b-82e6780a301e admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:42.162030 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69247f9e-ac24-40a2-b94b-82e6780a301e admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:42.222387 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1919f93-b572-4b55-b3bd-d19dcc90f264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:42.223058 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1919f93-b572-4b55-b3bd-d19dcc90f264 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:42.223423 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1919f93-b572-4b55-b3bd-d19dcc90f264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:42.223709 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1919f93-b572-4b55-b3bd-d19dcc90f264 admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:42.224390 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1919f93-b572-4b55-b3bd-d19dcc90f264 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:42.226327 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1919f93-b572-4b55-b3bd-d19dcc90f264 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:42.247944 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1006/2035] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:57:42 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:57:42.252147 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aa8cdc-4de7-491a-b0b5-5c11b4f0fd53 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:42.262643 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aa8cdc-4de7-491a-b0b5-5c11b4f0fd53 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:42.267544 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67574f6-ec85-43ab-b66c-5522c8f44667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:42.268191 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67574f6-ec85-43ab-b66c-5522c8f44667 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:42.268478 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67574f6-ec85-43ab-b66c-5522c8f44667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:42.268753 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67574f6-ec85-43ab-b66c-5522c8f44667 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:42.269497 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67574f6-ec85-43ab-b66c-5522c8f44667 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:42.271175 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67574f6-ec85-43ab-b66c-5522c8f44667 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:42.282041 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1030/2036] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:57:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:57:42.286139 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1919f93-b572-4b55-b3bd-d19dcc90f264 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:42.297116 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1919f93-b572-4b55-b3bd-d19dcc90f264 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:42.302430 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e167ff63-a751-487f-89c0-0f10fe6aa76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:42.303177 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167ff63-a751-487f-89c0-0f10fe6aa76c admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:42.303508 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167ff63-a751-487f-89c0-0f10fe6aa76c admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:42.303784 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167ff63-a751-487f-89c0-0f10fe6aa76c admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:42.304529 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e167ff63-a751-487f-89c0-0f10fe6aa76c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:42.306115 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e167ff63-a751-487f-89c0-0f10fe6aa76c admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:42.316349 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1007/2037] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:57:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:57:42.320577 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67574f6-ec85-43ab-b66c-5522c8f44667 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:42.330620 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67574f6-ec85-43ab-b66c-5522c8f44667 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:42.335522 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f284b32-484e-4b48-909e-acafe60d8863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:42.336436 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f284b32-484e-4b48-909e-acafe60d8863 admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:42.336718 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f284b32-484e-4b48-909e-acafe60d8863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:42.336996 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f284b32-484e-4b48-909e-acafe60d8863 admin admin] PATH_INFO: `/v3/projects/d1d8fd0d7e0a4dbfb59c8df13fd3b7b9/users/645e96c8b10c46d2a4198c23fbae84eb/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:42.353644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f284b32-484e-4b48-909e-acafe60d8863 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1d8fd0d7e0a4dbfb59c8df13fd3b7b9, user_id=645e96c8b10c46d2a4198c23fbae84eb, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:42.356243 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f284b32-484e-4b48-909e-acafe60d8863 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:42.372211 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1031/2038] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:57:42 2022] PUT /identity/v3/projects/d1d8fd0d7e0a4dbfb59c8df13fd3b7b9/users/645e96c8b10c46d2a4198c23fbae84eb/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:42.383332 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2c216-537c-49c1-ae8e-d7d73d35c71f None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:42.383653 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2c216-537c-49c1-ae8e-d7d73d35c71f None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:42.383901 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2c216-537c-49c1-ae8e-d7d73d35c71f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:42.415822 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-d2c2c216-537c-49c1-ae8e-d7d73d35c71f None None] MFA Rules not processed for user `645e96c8b10c46d2a4198c23fbae84eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:42.417131 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-d2c2c216-537c-49c1-ae8e-d7d73d35c71f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:42.455633 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:42.483339 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1008/2039] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:42.487784 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f284b32-484e-4b48-909e-acafe60d8863 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:42.542061 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:42.553743 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f284b32-484e-4b48-909e-acafe60d8863 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:42.560056 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a8cb4-b899-401b-a7a3-0bd89c6263f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:42.560716 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a8cb4-b899-401b-a7a3-0bd89c6263f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:42.560999 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a8cb4-b899-401b-a7a3-0bd89c6263f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:42.561276 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a8cb4-b899-401b-a7a3-0bd89c6263f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:42.561981 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4a8cb4-b899-401b-a7a3-0bd89c6263f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:42.563971 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4a8cb4-b899-401b-a7a3-0bd89c6263f4 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:42.591182 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1032/2040] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:57:42 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:57:42.595368 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c2c216-537c-49c1-ae8e-d7d73d35c71f None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:42.606542 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c2c216-537c-49c1-ae8e-d7d73d35c71f None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:42.661623 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f49bc-b714-4d51-b5db-23834a9f0631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:42.662323 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f49bc-b714-4d51-b5db-23834a9f0631 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:42.662604 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f49bc-b714-4d51-b5db-23834a9f0631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:42.663177 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f49bc-b714-4d51-b5db-23834a9f0631 admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:42.663592 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4f49bc-b714-4d51-b5db-23834a9f0631 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:42.665549 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4f49bc-b714-4d51-b5db-23834a9f0631 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:42.689479 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1009/2041] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:57:42 2022] POST /identity/v3/users => generated 298 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:57:42.693417 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a8cb4-b899-401b-a7a3-0bd89c6263f4 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:42.706918 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a8cb4-b899-401b-a7a3-0bd89c6263f4 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:42.715926 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd0fda9-ec79-4f34-a515-765b8ae8db08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:42.716591 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd0fda9-ec79-4f34-a515-765b8ae8db08 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:42.716874 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd0fda9-ec79-4f34-a515-765b8ae8db08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:42.717150 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd0fda9-ec79-4f34-a515-765b8ae8db08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:42.717889 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd0fda9-ec79-4f34-a515-765b8ae8db08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:42.719648 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd0fda9-ec79-4f34-a515-765b8ae8db08 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:42.733652 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1033/2042] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:57:42 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 01:57:42.737923 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f49bc-b714-4d51-b5db-23834a9f0631 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:42.748293 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f49bc-b714-4d51-b5db-23834a9f0631 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:42.753195 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1c485a-1dae-415d-988a-839b9cad8fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:42.754088 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1c485a-1dae-415d-988a-839b9cad8fe4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:42.754372 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1c485a-1dae-415d-988a-839b9cad8fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:42.754644 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1c485a-1dae-415d-988a-839b9cad8fe4 admin admin] PATH_INFO: `/v3/projects/2f2f1a1861994203894ab1bcd432fd83/users/2fe3057d0d3847a59ff159b5dd5768b2/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:42.770753 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1c485a-1dae-415d-988a-839b9cad8fe4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f2f1a1861994203894ab1bcd432fd83, user_id=2fe3057d0d3847a59ff159b5dd5768b2, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:42.773464 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1c485a-1dae-415d-988a-839b9cad8fe4 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:42.787600 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1010/2043] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:57:42 2022] PUT /identity/v3/projects/2f2f1a1861994203894ab1bcd432fd83/users/2fe3057d0d3847a59ff159b5dd5768b2/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 01:57:42.791558 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd0fda9-ec79-4f34-a515-765b8ae8db08 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:42.802350 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd0fda9-ec79-4f34-a515-765b8ae8db08 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:42.860853 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d160c6b3-1a7b-4318-9de3-6aedf6f12508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:42.861437 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d160c6b3-1a7b-4318-9de3-6aedf6f12508 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:42.861716 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d160c6b3-1a7b-4318-9de3-6aedf6f12508 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:42.861988 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d160c6b3-1a7b-4318-9de3-6aedf6f12508 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:42.862719 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d160c6b3-1a7b-4318-9de3-6aedf6f12508 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:42.864462 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d160c6b3-1a7b-4318-9de3-6aedf6f12508 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:42.874136 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1034/2044] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:57:42 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:57:42.878339 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1c485a-1dae-415d-988a-839b9cad8fe4 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:42.888665 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1c485a-1dae-415d-988a-839b9cad8fe4 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:42.893602 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81155e75-57f0-4248-82ba-0e78adaceb50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:42.894496 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81155e75-57f0-4248-82ba-0e78adaceb50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:42.894774 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81155e75-57f0-4248-82ba-0e78adaceb50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:42.895143 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81155e75-57f0-4248-82ba-0e78adaceb50 admin admin] PATH_INFO: `/v3/projects/2f2f1a1861994203894ab1bcd432fd83/users/2fe3057d0d3847a59ff159b5dd5768b2/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:42.897324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81155e75-57f0-4248-82ba-0e78adaceb50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f2f1a1861994203894ab1bcd432fd83, user_id=2fe3057d0d3847a59ff159b5dd5768b2, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:42.900011 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81155e75-57f0-4248-82ba-0e78adaceb50 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:42.910379 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1011/2045] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:57:42 2022] PUT /identity/v3/projects/2f2f1a1861994203894ab1bcd432fd83/users/2fe3057d0d3847a59ff159b5dd5768b2/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:42.920401 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed026bf-bc7a-438e-9ad8-f1ae42f5822b None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:42.920693 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed026bf-bc7a-438e-9ad8-f1ae42f5822b None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:42.920972 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed026bf-bc7a-438e-9ad8-f1ae42f5822b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:42.948698 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-2ed026bf-bc7a-438e-9ad8-f1ae42f5822b None None] MFA Rules not processed for user `2fe3057d0d3847a59ff159b5dd5768b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:42.949913 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-2ed026bf-bc7a-438e-9ad8-f1ae42f5822b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:43.024973 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1035/2046] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:42 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:43.038889 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81155e75-57f0-4248-82ba-0e78adaceb50 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:43.041360 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-81155e75-57f0-4248-82ba-0e78adaceb50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:43.103445 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81155e75-57f0-4248-82ba-0e78adaceb50 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:43.109460 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd5599e-1c89-4d8d-b45b-5e118f8ae91f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:43.110409 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5599e-1c89-4d8d-b45b-5e118f8ae91f service cinder] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:43.110679 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5599e-1c89-4d8d-b45b-5e118f8ae91f service cinder] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:43.111080 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5599e-1c89-4d8d-b45b-5e118f8ae91f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:43.125334 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd5599e-1c89-4d8d-b45b-5e118f8ae91f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:43.127608 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd5599e-1c89-4d8d-b45b-5e118f8ae91f service cinder] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:43.137624 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1012/2047] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:57:43 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:57:43.230287 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed026bf-bc7a-438e-9ad8-f1ae42f5822b None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:43.241159 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed026bf-bc7a-438e-9ad8-f1ae42f5822b None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:43.245563 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78045df0-886b-4d84-b1a8-de28fc4a9ddd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:43.246202 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78045df0-886b-4d84-b1a8-de28fc4a9ddd service cinder] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:43.246480 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78045df0-886b-4d84-b1a8-de28fc4a9ddd service cinder] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:43.246749 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78045df0-886b-4d84-b1a8-de28fc4a9ddd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:43.296406 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:43.310074 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78045df0-886b-4d84-b1a8-de28fc4a9ddd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:43.311621 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78045df0-886b-4d84-b1a8-de28fc4a9ddd service cinder] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:43.320722 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1036/2048] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:57:43 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:57:43.615026 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd5599e-1c89-4d8d-b45b-5e118f8ae91f service cinder] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:43.652612 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:43.681315 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd5599e-1c89-4d8d-b45b-5e118f8ae91f service cinder] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:43.686278 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796c9bfd-21dc-48fb-b26c-d51da4bba06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:43.687163 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c9bfd-21dc-48fb-b26c-d51da4bba06f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:43.687477 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c9bfd-21dc-48fb-b26c-d51da4bba06f admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:43.687754 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c9bfd-21dc-48fb-b26c-d51da4bba06f admin admin] PATH_INFO: `/v3/users/645e96c8b10c46d2a4198c23fbae84eb` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:43.689012 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796c9bfd-21dc-48fb-b26c-d51da4bba06f admin admin] RBAC: Authorizing `identity:delete_user(user_id=645e96c8b10c46d2a4198c23fbae84eb)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:43.690940 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796c9bfd-21dc-48fb-b26c-d51da4bba06f admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:43.744849 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-796c9bfd-21dc-48fb-b26c-d51da4bba06f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '645e96c8b10c46d2a4198c23fbae84eb'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:43.757532 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-796c9bfd-21dc-48fb-b26c-d51da4bba06f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '645e96c8b10c46d2a4198c23fbae84eb'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:43.767096 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-796c9bfd-21dc-48fb-b26c-d51da4bba06f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '645e96c8b10c46d2a4198c23fbae84eb'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:43.774292 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-796c9bfd-21dc-48fb-b26c-d51da4bba06f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '645e96c8b10c46d2a4198c23fbae84eb'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:43.774926 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-796c9bfd-21dc-48fb-b26c-d51da4bba06f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '645e96c8b10c46d2a4198c23fbae84eb'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:43.780792 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1013/2049] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:57:43 2022] DELETE /identity/v3/users/645e96c8b10c46d2a4198c23fbae84eb => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:43.785348 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b392b-2b0e-43ff-877b-e0befa1c1285 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:43.785769 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b392b-2b0e-43ff-877b-e0befa1c1285 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:43.786050 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b392b-2b0e-43ff-877b-e0befa1c1285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:43.808977 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-d97b392b-2b0e-43ff-877b-e0befa1c1285 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:43.810194 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-d97b392b-2b0e-43ff-877b-e0befa1c1285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:43.878154 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1037/2050] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:43.888194 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796c9bfd-21dc-48fb-b26c-d51da4bba06f admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:43.889701 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-796c9bfd-21dc-48fb-b26c-d51da4bba06f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:43.950658 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796c9bfd-21dc-48fb-b26c-d51da4bba06f admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:43.954820 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c869c4be-c063-4269-8419-db9f61cbeb5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:43.955511 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c869c4be-c063-4269-8419-db9f61cbeb5e service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:43.955795 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c869c4be-c063-4269-8419-db9f61cbeb5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:43.956071 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c869c4be-c063-4269-8419-db9f61cbeb5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:43.970401 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c869c4be-c063-4269-8419-db9f61cbeb5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:43.972632 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c869c4be-c063-4269-8419-db9f61cbeb5e service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:43.983492 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1014/2051] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:57:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:57:44.280803 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97b392b-2b0e-43ff-877b-e0befa1c1285 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:44.282389 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-d97b392b-2b0e-43ff-877b-e0befa1c1285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:44.294447 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97b392b-2b0e-43ff-877b-e0befa1c1285 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:44.299540 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c82de5b-8326-4aca-9ee7-355f7e31f265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:44.300434 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c82de5b-8326-4aca-9ee7-355f7e31f265 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:44.300717 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c82de5b-8326-4aca-9ee7-355f7e31f265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:44.300993 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c82de5b-8326-4aca-9ee7-355f7e31f265 admin admin] PATH_INFO: `/v3/projects/d1d8fd0d7e0a4dbfb59c8df13fd3b7b9` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:44.302526 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c82de5b-8326-4aca-9ee7-355f7e31f265 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1d8fd0d7e0a4dbfb59c8df13fd3b7b9)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:44.304628 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c82de5b-8326-4aca-9ee7-355f7e31f265 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:44.353506 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-0c82de5b-8326-4aca-9ee7-355f7e31f265 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1d8fd0d7e0a4dbfb59c8df13fd3b7b9', 'request_id': 'req-0c82de5b-8326-4aca-9ee7-355f7e31f265'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:44.359668 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-0c82de5b-8326-4aca-9ee7-355f7e31f265 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1d8fd0d7e0a4dbfb59c8df13fd3b7b9', 'request_id': 'req-0c82de5b-8326-4aca-9ee7-355f7e31f265'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:44.368208 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-0c82de5b-8326-4aca-9ee7-355f7e31f265 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1d8fd0d7e0a4dbfb59c8df13fd3b7b9', 'request_id': 'req-0c82de5b-8326-4aca-9ee7-355f7e31f265'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:44.378893 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-0c82de5b-8326-4aca-9ee7-355f7e31f265 admin admin] The token cache is being invalidate because project d1d8fd0d7e0a4dbfb59c8df13fd3b7b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:57:44.379232 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-0c82de5b-8326-4aca-9ee7-355f7e31f265 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:44.379876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-0c82de5b-8326-4aca-9ee7-355f7e31f265 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:44.381548 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1038/2052] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:57:44 2022] DELETE /identity/v3/projects/d1d8fd0d7e0a4dbfb59c8df13fd3b7b9 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:44.385520 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c869c4be-c063-4269-8419-db9f61cbeb5e service neutron] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:44.395707 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c869c4be-c063-4269-8419-db9f61cbeb5e service neutron] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:44.448303 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:44.456538 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea86c1e8-87ea-47ba-9440-6f695984b589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:44.457675 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea86c1e8-87ea-47ba-9440-6f695984b589 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:44.458212 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea86c1e8-87ea-47ba-9440-6f695984b589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:44.458508 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea86c1e8-87ea-47ba-9440-6f695984b589 admin admin] PATH_INFO: `/v3/users/2fe3057d0d3847a59ff159b5dd5768b2` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:44.459944 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea86c1e8-87ea-47ba-9440-6f695984b589 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fe3057d0d3847a59ff159b5dd5768b2)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:44.461944 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea86c1e8-87ea-47ba-9440-6f695984b589 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:44.512932 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-ea86c1e8-87ea-47ba-9440-6f695984b589 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fe3057d0d3847a59ff159b5dd5768b2'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:44.525096 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-ea86c1e8-87ea-47ba-9440-6f695984b589 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fe3057d0d3847a59ff159b5dd5768b2'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:44.534624 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-ea86c1e8-87ea-47ba-9440-6f695984b589 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fe3057d0d3847a59ff159b5dd5768b2'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:44.541673 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-ea86c1e8-87ea-47ba-9440-6f695984b589 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fe3057d0d3847a59ff159b5dd5768b2'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:44.542413 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-ea86c1e8-87ea-47ba-9440-6f695984b589 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fe3057d0d3847a59ff159b5dd5768b2'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:44.550202 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1015/2053] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:57:44 2022] DELETE /identity/v3/users/2fe3057d0d3847a59ff159b5dd5768b2 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:44.895593 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c82de5b-8326-4aca-9ee7-355f7e31f265 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:44.897566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-0c82de5b-8326-4aca-9ee7-355f7e31f265 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:44.919587 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c82de5b-8326-4aca-9ee7-355f7e31f265 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:44.978115 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa028562-7467-4587-99a4-c046cb0fe15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:44.978980 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa028562-7467-4587-99a4-c046cb0fe15a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:44.979291 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa028562-7467-4587-99a4-c046cb0fe15a admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:44.979568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa028562-7467-4587-99a4-c046cb0fe15a admin admin] PATH_INFO: `/v3/projects/2f2f1a1861994203894ab1bcd432fd83` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:44.980853 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa028562-7467-4587-99a4-c046cb0fe15a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f2f1a1861994203894ab1bcd432fd83)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:44.982794 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa028562-7467-4587-99a4-c046cb0fe15a admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:44.994969 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:45.050899 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-fa028562-7467-4587-99a4-c046cb0fe15a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f2f1a1861994203894ab1bcd432fd83', 'request_id': 'req-fa028562-7467-4587-99a4-c046cb0fe15a'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:45.057552 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-fa028562-7467-4587-99a4-c046cb0fe15a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f2f1a1861994203894ab1bcd432fd83', 'request_id': 'req-fa028562-7467-4587-99a4-c046cb0fe15a'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:45.066638 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-fa028562-7467-4587-99a4-c046cb0fe15a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f2f1a1861994203894ab1bcd432fd83', 'request_id': 'req-fa028562-7467-4587-99a4-c046cb0fe15a'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:45.076931 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-fa028562-7467-4587-99a4-c046cb0fe15a admin admin] The token cache is being invalidate because project 2f2f1a1861994203894ab1bcd432fd83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:57:45.077218 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-fa028562-7467-4587-99a4-c046cb0fe15a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:45.077921 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-fa028562-7467-4587-99a4-c046cb0fe15a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:45.079559 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1039/2054] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:57:44 2022] DELETE /identity/v3/projects/2f2f1a1861994203894ab1bcd432fd83 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:45.084956 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4835bbf-9f10-4255-b77f-6e72c6acd254 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:45.085245 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4835bbf-9f10-4255-b77f-6e72c6acd254 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:45.085524 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4835bbf-9f10-4255-b77f-6e72c6acd254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:45.105803 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-d4835bbf-9f10-4255-b77f-6e72c6acd254 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:45.107105 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-d4835bbf-9f10-4255-b77f-6e72c6acd254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:45.172424 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1016/2055] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 01:57:45.181623 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5e4f5-9570-4c04-94c6-2fd72b68c991 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:45.181915 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5e4f5-9570-4c04-94c6-2fd72b68c991 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:45.182201 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5e4f5-9570-4c04-94c6-2fd72b68c991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:45.202280 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-d2b5e4f5-9570-4c04-94c6-2fd72b68c991 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:45.203526 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-d2b5e4f5-9570-4c04-94c6-2fd72b68c991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:45.213052 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1040/2056] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:45.217476 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4835bbf-9f10-4255-b77f-6e72c6acd254 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:45.228027 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4835bbf-9f10-4255-b77f-6e72c6acd254 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:45.233395 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5988f4-6512-4e67-9824-ba63b48618cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:45.234051 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5988f4-6512-4e67-9824-ba63b48618cb admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:45.234338 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5988f4-6512-4e67-9824-ba63b48618cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:45.234607 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5988f4-6512-4e67-9824-ba63b48618cb admin admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:45.235368 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5988f4-6512-4e67-9824-ba63b48618cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:57:45.235696 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5988f4-6512-4e67-9824-ba63b48618cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:45.237281 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5988f4-6512-4e67-9824-ba63b48618cb admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:45.249101 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1017/2057] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 01:57:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:57:45.253940 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b5e4f5-9570-4c04-94c6-2fd72b68c991 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:45.263817 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b5e4f5-9570-4c04-94c6-2fd72b68c991 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:45.269203 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5901d2-1b42-4cd4-a7e7-83d17d4f39d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:45.269853 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5901d2-1b42-4cd4-a7e7-83d17d4f39d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:45.270131 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5901d2-1b42-4cd4-a7e7-83d17d4f39d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:45.270405 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5901d2-1b42-4cd4-a7e7-83d17d4f39d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:45.271116 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5901d2-1b42-4cd4-a7e7-83d17d4f39d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:45.273049 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5901d2-1b42-4cd4-a7e7-83d17d4f39d9 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:45.300442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1041/2058] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:57:45 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:57:45.304611 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5988f4-6512-4e67-9824-ba63b48618cb admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:45.314741 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5988f4-6512-4e67-9824-ba63b48618cb admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:45.372385 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888556ac-5e8c-4bf6-a2af-5ee1adcbb608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:45.373046 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888556ac-5e8c-4bf6-a2af-5ee1adcbb608 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:45.373323 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888556ac-5e8c-4bf6-a2af-5ee1adcbb608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:45.373595 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888556ac-5e8c-4bf6-a2af-5ee1adcbb608 admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:45.374299 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888556ac-5e8c-4bf6-a2af-5ee1adcbb608 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:45.376233 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888556ac-5e8c-4bf6-a2af-5ee1adcbb608 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:45.383823 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:45.397392 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1018/2059] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:57:45 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:57:45.400841 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5901d2-1b42-4cd4-a7e7-83d17d4f39d9 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:45.411904 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5901d2-1b42-4cd4-a7e7-83d17d4f39d9 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:45.417771 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734d6c05-07ef-4ed1-9543-f057b60c514d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:45.418425 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734d6c05-07ef-4ed1-9543-f057b60c514d admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:45.418705 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734d6c05-07ef-4ed1-9543-f057b60c514d admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:45.418979 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734d6c05-07ef-4ed1-9543-f057b60c514d admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:45.419760 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734d6c05-07ef-4ed1-9543-f057b60c514d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:45.421384 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734d6c05-07ef-4ed1-9543-f057b60c514d admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:45.432590 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1042/2060] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:57:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:57:45.436584 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888556ac-5e8c-4bf6-a2af-5ee1adcbb608 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:45.447151 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888556ac-5e8c-4bf6-a2af-5ee1adcbb608 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:45.452382 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f043a8a-4df3-442d-98c8-42e65829cdaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:45.453040 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f043a8a-4df3-442d-98c8-42e65829cdaa admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:45.453352 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f043a8a-4df3-442d-98c8-42e65829cdaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:45.453621 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f043a8a-4df3-442d-98c8-42e65829cdaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:45.454340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f043a8a-4df3-442d-98c8-42e65829cdaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:45.455985 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f043a8a-4df3-442d-98c8-42e65829cdaa admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:45.466083 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1019/2061] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:57:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:57:45.470331 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734d6c05-07ef-4ed1-9543-f057b60c514d admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:45.480948 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734d6c05-07ef-4ed1-9543-f057b60c514d admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:45.485724 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5e2aae-e22e-4686-b2cd-cf7d89bc6db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:45.486590 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e2aae-e22e-4686-b2cd-cf7d89bc6db3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:45.486870 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e2aae-e22e-4686-b2cd-cf7d89bc6db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:45.487139 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e2aae-e22e-4686-b2cd-cf7d89bc6db3 admin admin] PATH_INFO: `/v3/projects/98b4b14a19814617a0fb97b7de7ba1a5/users/099d52d1c6144833a15335f2fcac597b/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:45.501918 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5e2aae-e22e-4686-b2cd-cf7d89bc6db3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98b4b14a19814617a0fb97b7de7ba1a5, user_id=099d52d1c6144833a15335f2fcac597b, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:45.504679 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5e2aae-e22e-4686-b2cd-cf7d89bc6db3 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:45.519555 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1043/2062] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:57:45 2022] PUT /identity/v3/projects/98b4b14a19814617a0fb97b7de7ba1a5/users/099d52d1c6144833a15335f2fcac597b/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:45.529802 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d59ad1-4d3c-4f84-8ace-f6e7f9d2067b None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:45.530162 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d59ad1-4d3c-4f84-8ace-f6e7f9d2067b None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:45.530565 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d59ad1-4d3c-4f84-8ace-f6e7f9d2067b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:45.559195 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-49d59ad1-4d3c-4f84-8ace-f6e7f9d2067b None None] MFA Rules not processed for user `099d52d1c6144833a15335f2fcac597b`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:45.560527 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-49d59ad1-4d3c-4f84-8ace-f6e7f9d2067b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:45.629854 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1020/2063] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:45.641478 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5e2aae-e22e-4686-b2cd-cf7d89bc6db3 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:45.644420 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-9d5e2aae-e22e-4686-b2cd-cf7d89bc6db3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:45.712383 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5e2aae-e22e-4686-b2cd-cf7d89bc6db3 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:45.716275 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4d5dc5-594b-4b0a-9bf8-f3e9790f1683 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:45.716917 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d5dc5-594b-4b0a-9bf8-f3e9790f1683 service cinder] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:45.717205 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d5dc5-594b-4b0a-9bf8-f3e9790f1683 service cinder] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:45.717476 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d5dc5-594b-4b0a-9bf8-f3e9790f1683 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:45.727801 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4d5dc5-594b-4b0a-9bf8-f3e9790f1683 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:45.730127 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4d5dc5-594b-4b0a-9bf8-f3e9790f1683 service cinder] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:45.742887 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1044/2064] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:57:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:57:45.763659 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d59ad1-4d3c-4f84-8ace-f6e7f9d2067b None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:45.830904 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d59ad1-4d3c-4f84-8ace-f6e7f9d2067b None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:45.837180 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb89dab-44a8-414c-8abe-39b32097f524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:45.838071 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb89dab-44a8-414c-8abe-39b32097f524 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:45.838386 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb89dab-44a8-414c-8abe-39b32097f524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:45.838608 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb89dab-44a8-414c-8abe-39b32097f524 admin admin] PATH_INFO: `/v3/users/099d52d1c6144833a15335f2fcac597b` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:45.840025 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb89dab-44a8-414c-8abe-39b32097f524 admin admin] RBAC: Authorizing `identity:delete_user(user_id=099d52d1c6144833a15335f2fcac597b)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:45.842090 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb89dab-44a8-414c-8abe-39b32097f524 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:45.904253 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-aeb89dab-44a8-414c-8abe-39b32097f524 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '099d52d1c6144833a15335f2fcac597b'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:45.918665 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-aeb89dab-44a8-414c-8abe-39b32097f524 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '099d52d1c6144833a15335f2fcac597b'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:45.930083 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-aeb89dab-44a8-414c-8abe-39b32097f524 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '099d52d1c6144833a15335f2fcac597b'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:45.931833 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:45.940562 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-aeb89dab-44a8-414c-8abe-39b32097f524 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '099d52d1c6144833a15335f2fcac597b'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:45.941319 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-aeb89dab-44a8-414c-8abe-39b32097f524 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '099d52d1c6144833a15335f2fcac597b'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:45.952375 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1021/2065] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:57:45 2022] DELETE /identity/v3/users/099d52d1c6144833a15335f2fcac597b => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:45.958386 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc9516-74dc-461c-ac58-4930be8b8993 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:45.959627 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc9516-74dc-461c-ac58-4930be8b8993 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:45.960019 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc9516-74dc-461c-ac58-4930be8b8993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:45.983259 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-3dcc9516-74dc-461c-ac58-4930be8b8993 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:45.984875 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-3dcc9516-74dc-461c-ac58-4930be8b8993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:46.027178 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:46.066784 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1045/2066] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:46.078604 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb89dab-44a8-414c-8abe-39b32097f524 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:46.080163 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-aeb89dab-44a8-414c-8abe-39b32097f524 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:46.145488 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb89dab-44a8-414c-8abe-39b32097f524 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:46.150463 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c742e62-179e-4f00-a127-7ba121b005b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:46.151263 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c742e62-179e-4f00-a127-7ba121b005b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:46.151672 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c742e62-179e-4f00-a127-7ba121b005b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:46.152055 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c742e62-179e-4f00-a127-7ba121b005b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:46.164792 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c742e62-179e-4f00-a127-7ba121b005b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:46.167013 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c742e62-179e-4f00-a127-7ba121b005b3 service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:46.178767 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1022/2067] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:57:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:57:46.491459 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc9516-74dc-461c-ac58-4930be8b8993 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:46.493521 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-3dcc9516-74dc-461c-ac58-4930be8b8993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:46.510797 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc9516-74dc-461c-ac58-4930be8b8993 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:46.516014 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b2f3bb-93ed-44a6-b25b-40b493e8168d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:46.516975 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2f3bb-93ed-44a6-b25b-40b493e8168d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:46.517157 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2f3bb-93ed-44a6-b25b-40b493e8168d admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:46.517581 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2f3bb-93ed-44a6-b25b-40b493e8168d admin admin] PATH_INFO: `/v3/projects/98b4b14a19814617a0fb97b7de7ba1a5` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:46.518846 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b2f3bb-93ed-44a6-b25b-40b493e8168d admin admin] RBAC: Authorizing `identity:delete_project(project_id=98b4b14a19814617a0fb97b7de7ba1a5)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:46.520953 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b2f3bb-93ed-44a6-b25b-40b493e8168d admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:46.575123 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-65b2f3bb-93ed-44a6-b25b-40b493e8168d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98b4b14a19814617a0fb97b7de7ba1a5', 'request_id': 'req-65b2f3bb-93ed-44a6-b25b-40b493e8168d'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:46.581813 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-65b2f3bb-93ed-44a6-b25b-40b493e8168d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98b4b14a19814617a0fb97b7de7ba1a5', 'request_id': 'req-65b2f3bb-93ed-44a6-b25b-40b493e8168d'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:46.592795 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-65b2f3bb-93ed-44a6-b25b-40b493e8168d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98b4b14a19814617a0fb97b7de7ba1a5', 'request_id': 'req-65b2f3bb-93ed-44a6-b25b-40b493e8168d'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:46.603615 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-65b2f3bb-93ed-44a6-b25b-40b493e8168d admin admin] The token cache is being invalidate because project 98b4b14a19814617a0fb97b7de7ba1a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:57:46.603903 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-65b2f3bb-93ed-44a6-b25b-40b493e8168d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:46.604531 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-65b2f3bb-93ed-44a6-b25b-40b493e8168d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:57:46.606117 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1046/2068] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:57:46 2022] DELETE /identity/v3/projects/98b4b14a19814617a0fb97b7de7ba1a5 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:46.611357 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a5b2c-69d1-4d76-8a66-e1c6ca36ab2b None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:46.611841 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a5b2c-69d1-4d76-8a66-e1c6ca36ab2b None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:46.611935 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a5b2c-69d1-4d76-8a66-e1c6ca36ab2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:46.631656 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-038a5b2c-69d1-4d76-8a66-e1c6ca36ab2b None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:46.631656 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-038a5b2c-69d1-4d76-8a66-e1c6ca36ab2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:46.697486 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1023/2069] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:46.709371 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d4de1-d04f-4d08-9416-be47e316e585 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:46.709828 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d4de1-d04f-4d08-9416-be47e316e585 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:46.710264 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d4de1-d04f-4d08-9416-be47e316e585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:46.735887 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-ec7d4de1-d04f-4d08-9416-be47e316e585 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:46.737028 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-ec7d4de1-d04f-4d08-9416-be47e316e585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:46.747444 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1047/2070] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:46.754130 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038a5b2c-69d1-4d76-8a66-e1c6ca36ab2b None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:46.764378 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038a5b2c-69d1-4d76-8a66-e1c6ca36ab2b None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:46.770173 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5785f6-3b76-4ced-aa65-3f7e258310e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:46.770834 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5785f6-3b76-4ced-aa65-3f7e258310e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:46.771111 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5785f6-3b76-4ced-aa65-3f7e258310e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:46.771409 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5785f6-3b76-4ced-aa65-3f7e258310e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:46.772198 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5785f6-3b76-4ced-aa65-3f7e258310e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:57:46.772504 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5785f6-3b76-4ced-aa65-3f7e258310e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:46.774114 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5785f6-3b76-4ced-aa65-3f7e258310e4 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:46.787308 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1024/2071] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 01:57:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:57:46.791759 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d4de1-d04f-4d08-9416-be47e316e585 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:46.801773 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d4de1-d04f-4d08-9416-be47e316e585 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:46.807776 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ee7a55-894b-41be-b497-6dadb2193176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:46.808423 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee7a55-894b-41be-b497-6dadb2193176 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:46.808701 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee7a55-894b-41be-b497-6dadb2193176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:46.808970 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee7a55-894b-41be-b497-6dadb2193176 admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:46.809724 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ee7a55-894b-41be-b497-6dadb2193176 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:46.811691 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ee7a55-894b-41be-b497-6dadb2193176 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:46.840137 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1048/2072] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:57:46 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:57:46.844294 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5785f6-3b76-4ced-aa65-3f7e258310e4 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:46.856661 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5785f6-3b76-4ced-aa65-3f7e258310e4 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:46.925005 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b4434-72e1-4c99-9f83-ffec719863ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:46.925668 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b4434-72e1-4c99-9f83-ffec719863ca admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:46.925958 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b4434-72e1-4c99-9f83-ffec719863ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:46.926226 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b4434-72e1-4c99-9f83-ffec719863ca admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:46.926921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798b4434-72e1-4c99-9f83-ffec719863ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:46.928910 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798b4434-72e1-4c99-9f83-ffec719863ca admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:46.950828 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1025/2073] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:57:46 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:57:46.955105 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ee7a55-894b-41be-b497-6dadb2193176 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:46.965936 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ee7a55-894b-41be-b497-6dadb2193176 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:46.970911 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd4f361-4182-4e53-875e-ffc5ca0921d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:46.971599 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd4f361-4182-4e53-875e-ffc5ca0921d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:46.971909 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd4f361-4182-4e53-875e-ffc5ca0921d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:46.972184 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd4f361-4182-4e53-875e-ffc5ca0921d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:46.972945 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd4f361-4182-4e53-875e-ffc5ca0921d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:46.974592 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd4f361-4182-4e53-875e-ffc5ca0921d1 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:46.985280 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1049/2074] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:57:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:57:46.989092 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b4434-72e1-4c99-9f83-ffec719863ca admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:46.999042 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b4434-72e1-4c99-9f83-ffec719863ca admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:47.005344 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c43f72c-8eb4-40fb-8919-bb8b176ea358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:47.006094 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c43f72c-8eb4-40fb-8919-bb8b176ea358 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:47.006377 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c43f72c-8eb4-40fb-8919-bb8b176ea358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:47.006646 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c43f72c-8eb4-40fb-8919-bb8b176ea358 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:47.007431 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c43f72c-8eb4-40fb-8919-bb8b176ea358 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:47.009072 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c43f72c-8eb4-40fb-8919-bb8b176ea358 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:47.011366 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:47.025193 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1026/2075] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:57:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 01:57:47.026928 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd4f361-4182-4e53-875e-ffc5ca0921d1 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:47.037353 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd4f361-4182-4e53-875e-ffc5ca0921d1 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:47.044942 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d568ccc-34ec-41b2-9fa1-185ce562fe49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:47.045837 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d568ccc-34ec-41b2-9fa1-185ce562fe49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:47.046112 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d568ccc-34ec-41b2-9fa1-185ce562fe49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:47.046389 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d568ccc-34ec-41b2-9fa1-185ce562fe49 admin admin] PATH_INFO: `/v3/projects/9a8f1ba000cf4d62a1be41232bf73eaa/users/7777652a2360413daea7c0a5eeb10d02/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:47.063598 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d568ccc-34ec-41b2-9fa1-185ce562fe49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a8f1ba000cf4d62a1be41232bf73eaa, user_id=7777652a2360413daea7c0a5eeb10d02, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:47.066409 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d568ccc-34ec-41b2-9fa1-185ce562fe49 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:47.082156 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1050/2076] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:57:47 2022] PUT /identity/v3/projects/9a8f1ba000cf4d62a1be41232bf73eaa/users/7777652a2360413daea7c0a5eeb10d02/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:47.093072 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4887d8f-e9e7-4fae-9a25-0f5f53c1197b None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:47.093331 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4887d8f-e9e7-4fae-9a25-0f5f53c1197b None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:47.093608 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4887d8f-e9e7-4fae-9a25-0f5f53c1197b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:47.121459 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-d4887d8f-e9e7-4fae-9a25-0f5f53c1197b None None] MFA Rules not processed for user `7777652a2360413daea7c0a5eeb10d02`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:47.122673 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-d4887d8f-e9e7-4fae-9a25-0f5f53c1197b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:47.191231 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1027/2077] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:47.195708 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d568ccc-34ec-41b2-9fa1-185ce562fe49 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:47.259791 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d568ccc-34ec-41b2-9fa1-185ce562fe49 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:47.265871 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ab0368-5537-4688-af16-de03ec1b6549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:47.266629 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab0368-5537-4688-af16-de03ec1b6549 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:47.266981 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab0368-5537-4688-af16-de03ec1b6549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:47.267385 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab0368-5537-4688-af16-de03ec1b6549 admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:47.268166 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ab0368-5537-4688-af16-de03ec1b6549 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:47.270925 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ab0368-5537-4688-af16-de03ec1b6549 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:47.274769 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:47.303864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1051/2078] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:57:47 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:57:47.308022 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4887d8f-e9e7-4fae-9a25-0f5f53c1197b None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:47.318857 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4887d8f-e9e7-4fae-9a25-0f5f53c1197b None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:47.378877 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298be31f-52fa-4e54-b474-91a4be36bdd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:47.379580 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298be31f-52fa-4e54-b474-91a4be36bdd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:47.379862 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298be31f-52fa-4e54-b474-91a4be36bdd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:47.380138 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298be31f-52fa-4e54-b474-91a4be36bdd9 admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:47.380830 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298be31f-52fa-4e54-b474-91a4be36bdd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:47.382758 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298be31f-52fa-4e54-b474-91a4be36bdd9 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:47.403584 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1028/2079] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:57:47 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:57:47.407722 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ab0368-5537-4688-af16-de03ec1b6549 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:47.418403 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ab0368-5537-4688-af16-de03ec1b6549 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:47.424296 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d5634-16b2-4ce2-bbe1-81a1a7bfe928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:47.424954 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d5634-16b2-4ce2-bbe1-81a1a7bfe928 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:47.425238 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d5634-16b2-4ce2-bbe1-81a1a7bfe928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:47.425513 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d5634-16b2-4ce2-bbe1-81a1a7bfe928 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:47.426336 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050d5634-16b2-4ce2-bbe1-81a1a7bfe928 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:47.428090 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050d5634-16b2-4ce2-bbe1-81a1a7bfe928 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:47.439000 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1052/2080] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:57:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:57:47.443026 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298be31f-52fa-4e54-b474-91a4be36bdd9 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:47.453469 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298be31f-52fa-4e54-b474-91a4be36bdd9 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:47.459585 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779175ff-c800-43c6-8d69-f779985dfb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:47.460245 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779175ff-c800-43c6-8d69-f779985dfb3a admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:47.460523 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779175ff-c800-43c6-8d69-f779985dfb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:47.460795 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779175ff-c800-43c6-8d69-f779985dfb3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:47.461529 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779175ff-c800-43c6-8d69-f779985dfb3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:47.463121 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779175ff-c800-43c6-8d69-f779985dfb3a admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:47.473186 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1029/2081] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:57:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:57:47.477447 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d5634-16b2-4ce2-bbe1-81a1a7bfe928 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:47.487653 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d5634-16b2-4ce2-bbe1-81a1a7bfe928 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:47.492826 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d706fa6-4584-46a6-89ae-537a1adc9f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:47.493749 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d706fa6-4584-46a6-89ae-537a1adc9f68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:47.494027 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d706fa6-4584-46a6-89ae-537a1adc9f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:47.494301 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d706fa6-4584-46a6-89ae-537a1adc9f68 admin admin] PATH_INFO: `/v3/projects/be90eb0787ce44bf923c70b10b72c01f/users/50f030020aba4f45b61582387495de6c/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:47.509795 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d706fa6-4584-46a6-89ae-537a1adc9f68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be90eb0787ce44bf923c70b10b72c01f, user_id=50f030020aba4f45b61582387495de6c, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:47.512433 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d706fa6-4584-46a6-89ae-537a1adc9f68 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:47.527377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1053/2082] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:57:47 2022] PUT /identity/v3/projects/be90eb0787ce44bf923c70b10b72c01f/users/50f030020aba4f45b61582387495de6c/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:47.537717 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d0d509-3c07-47cb-91fc-dbb9e197a398 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:47.538072 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d0d509-3c07-47cb-91fc-dbb9e197a398 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:47.538355 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d0d509-3c07-47cb-91fc-dbb9e197a398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:47.566946 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-e0d0d509-3c07-47cb-91fc-dbb9e197a398 None None] MFA Rules not processed for user `50f030020aba4f45b61582387495de6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:47.568237 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-e0d0d509-3c07-47cb-91fc-dbb9e197a398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:47.627373 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:47.639022 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1030/2083] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:47.643383 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d706fa6-4584-46a6-89ae-537a1adc9f68 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:47.708490 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d706fa6-4584-46a6-89ae-537a1adc9f68 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:47.714056 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8897d87-c420-4fc2-8b77-52ac991fe42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:47.714710 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8897d87-c420-4fc2-8b77-52ac991fe42b admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:47.714991 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8897d87-c420-4fc2-8b77-52ac991fe42b admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:47.715303 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8897d87-c420-4fc2-8b77-52ac991fe42b admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:47.715997 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8897d87-c420-4fc2-8b77-52ac991fe42b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:47.717907 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8897d87-c420-4fc2-8b77-52ac991fe42b admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:47.746317 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1054/2084] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:57:47 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:57:47.750730 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d0d509-3c07-47cb-91fc-dbb9e197a398 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:47.761999 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d0d509-3c07-47cb-91fc-dbb9e197a398 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:47.826792 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a4d36b-1d52-4958-8c5d-d0a1979bef94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:47.827498 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4d36b-1d52-4958-8c5d-d0a1979bef94 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:47.827779 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4d36b-1d52-4958-8c5d-d0a1979bef94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:47.828051 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4d36b-1d52-4958-8c5d-d0a1979bef94 admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:47.828775 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a4d36b-1d52-4958-8c5d-d0a1979bef94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:47.830750 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a4d36b-1d52-4958-8c5d-d0a1979bef94 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:47.855341 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1031/2085] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:57:47 2022] POST /identity/v3/users => generated 299 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:57:47.859424 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8897d87-c420-4fc2-8b77-52ac991fe42b admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:47.870133 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8897d87-c420-4fc2-8b77-52ac991fe42b admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:47.875374 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c3397a-0218-49b9-8dcf-bf4f426e046f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:47.876059 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c3397a-0218-49b9-8dcf-bf4f426e046f admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:47.876340 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c3397a-0218-49b9-8dcf-bf4f426e046f admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:47.876619 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c3397a-0218-49b9-8dcf-bf4f426e046f admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:47.877352 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c3397a-0218-49b9-8dcf-bf4f426e046f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:47.878974 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c3397a-0218-49b9-8dcf-bf4f426e046f admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:47.889938 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1055/2086] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:57:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:57:47.893951 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a4d36b-1d52-4958-8c5d-d0a1979bef94 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:47.904345 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a4d36b-1d52-4958-8c5d-d0a1979bef94 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:47.910473 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada0b23f-cdfc-4160-bfd6-f4e37180fa70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:47.911657 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0b23f-cdfc-4160-bfd6-f4e37180fa70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:47.911937 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0b23f-cdfc-4160-bfd6-f4e37180fa70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:47.912209 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0b23f-cdfc-4160-bfd6-f4e37180fa70 admin admin] PATH_INFO: `/v3/projects/a6dc5009e9e24dbc8b8cea2436e35f5d/users/41fb9db1b90f471ba60a8897546ded68/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:47.929342 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada0b23f-cdfc-4160-bfd6-f4e37180fa70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6dc5009e9e24dbc8b8cea2436e35f5d, user_id=41fb9db1b90f471ba60a8897546ded68, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:47.932048 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada0b23f-cdfc-4160-bfd6-f4e37180fa70 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:47.947227 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1032/2087] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:57:47 2022] PUT /identity/v3/projects/a6dc5009e9e24dbc8b8cea2436e35f5d/users/41fb9db1b90f471ba60a8897546ded68/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:47.951624 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c3397a-0218-49b9-8dcf-bf4f426e046f admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:47.963552 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c3397a-0218-49b9-8dcf-bf4f426e046f admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:48.025179 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496efed3-fe07-40b4-af61-ef85e930508f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:48.025803 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496efed3-fe07-40b4-af61-ef85e930508f admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:48.026082 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496efed3-fe07-40b4-af61-ef85e930508f admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:48.026360 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496efed3-fe07-40b4-af61-ef85e930508f admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:48.027122 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496efed3-fe07-40b4-af61-ef85e930508f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:48.028880 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496efed3-fe07-40b4-af61-ef85e930508f admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:48.039490 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1056/2088] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:57:47 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:57:48.043916 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada0b23f-cdfc-4160-bfd6-f4e37180fa70 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:48.055105 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada0b23f-cdfc-4160-bfd6-f4e37180fa70 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:48.060207 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd74b58-8565-4d78-9a36-a1ba5161302e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:48.061095 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd74b58-8565-4d78-9a36-a1ba5161302e admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:48.061370 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd74b58-8565-4d78-9a36-a1ba5161302e admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:48.061641 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd74b58-8565-4d78-9a36-a1ba5161302e admin admin] PATH_INFO: `/v3/projects/a6dc5009e9e24dbc8b8cea2436e35f5d/users/41fb9db1b90f471ba60a8897546ded68/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:48.063817 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd74b58-8565-4d78-9a36-a1ba5161302e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6dc5009e9e24dbc8b8cea2436e35f5d, user_id=41fb9db1b90f471ba60a8897546ded68, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:48.066479 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd74b58-8565-4d78-9a36-a1ba5161302e admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:48.078075 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1033/2089] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:57:48 2022] PUT /identity/v3/projects/a6dc5009e9e24dbc8b8cea2436e35f5d/users/41fb9db1b90f471ba60a8897546ded68/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:57:48.089454 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222a4b1-f626-472c-a55d-b83265bf044d None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:48.089748 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222a4b1-f626-472c-a55d-b83265bf044d None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:48.090025 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222a4b1-f626-472c-a55d-b83265bf044d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:48.118530 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-4222a4b1-f626-472c-a55d-b83265bf044d None None] MFA Rules not processed for user `41fb9db1b90f471ba60a8897546ded68`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:57:48.119798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-4222a4b1-f626-472c-a55d-b83265bf044d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:48.128777 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:48.191466 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1057/2090] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:57:48 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:57:48.202889 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd74b58-8565-4d78-9a36-a1ba5161302e admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:48.204926 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-edd74b58-8565-4d78-9a36-a1ba5161302e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:48.263524 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd74b58-8565-4d78-9a36-a1ba5161302e admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:48.267402 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548893fa-b6fb-4123-b5f6-2db6f7b4b36f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:48.268212 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548893fa-b6fb-4123-b5f6-2db6f7b4b36f service cinder] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:48.268490 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548893fa-b6fb-4123-b5f6-2db6f7b4b36f service cinder] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:48.268765 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548893fa-b6fb-4123-b5f6-2db6f7b4b36f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:48.335602 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548893fa-b6fb-4123-b5f6-2db6f7b4b36f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:48.336952 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548893fa-b6fb-4123-b5f6-2db6f7b4b36f service cinder] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:48.346614 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1034/2091] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:57:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:57:54.072035 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222a4b1-f626-472c-a55d-b83265bf044d None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:54.073632 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-4222a4b1-f626-472c-a55d-b83265bf044d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:57:54.086148 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222a4b1-f626-472c-a55d-b83265bf044d None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:54.089686 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7e6c20-ae9c-4c40-b53a-452f310f848a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:54.090321 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e6c20-ae9c-4c40-b53a-452f310f848a service cinder] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:54.090599 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e6c20-ae9c-4c40-b53a-452f310f848a service cinder] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:54.090872 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e6c20-ae9c-4c40-b53a-452f310f848a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:54.151739 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7e6c20-ae9c-4c40-b53a-452f310f848a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:54.152729 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7e6c20-ae9c-4c40-b53a-452f310f848a service cinder] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:54.162499 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1058/2092] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:57:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:57:54.750151 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548893fa-b6fb-4123-b5f6-2db6f7b4b36f service cinder] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:57:54.754294 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:57:54.760242 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548893fa-b6fb-4123-b5f6-2db6f7b4b36f service cinder] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:57:54.764273 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f1c99d-60d6-4e32-b6ae-6695e7d97851 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:57:54.764916 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f1c99d-60d6-4e32-b6ae-6695e7d97851 service cinder] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:57:54.765197 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f1c99d-60d6-4e32-b6ae-6695e7d97851 service cinder] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:57:54.765473 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f1c99d-60d6-4e32-b6ae-6695e7d97851 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:57:54.774427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f1c99d-60d6-4e32-b6ae-6695e7d97851 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:57:54.775894 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f1c99d-60d6-4e32-b6ae-6695e7d97851 service cinder] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:57:54.785657 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1035/2093] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:57:54 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:58:04.158006 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=53 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:04.781266 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=5 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:06.099511 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7e6c20-ae9c-4c40-b53a-452f310f848a service cinder] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:06.167047 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f1c99d-60d6-4e32-b6ae-6695e7d97851 service cinder] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:06.169018 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-25f1c99d-60d6-4e32-b6ae-6695e7d97851 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:06.172837 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7e6c20-ae9c-4c40-b53a-452f310f848a service cinder] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:06.178032 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9221820c-b4b0-46d3-b30b-0fdc3d0170f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:06.179253 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9221820c-b4b0-46d3-b30b-0fdc3d0170f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:06.179749 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9221820c-b4b0-46d3-b30b-0fdc3d0170f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:06.180113 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9221820c-b4b0-46d3-b30b-0fdc3d0170f7 admin admin] PATH_INFO: `/v3/users/7777652a2360413daea7c0a5eeb10d02` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:06.183438 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9221820c-b4b0-46d3-b30b-0fdc3d0170f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7777652a2360413daea7c0a5eeb10d02)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:06.186134 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9221820c-b4b0-46d3-b30b-0fdc3d0170f7 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:06.187745 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f1c99d-60d6-4e32-b6ae-6695e7d97851 service cinder] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:06.192722 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f9e718-7e55-46c9-9827-46cd3f5aba62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:06.193630 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9e718-7e55-46c9-9827-46cd3f5aba62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:06.193907 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9e718-7e55-46c9-9827-46cd3f5aba62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:06.194177 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9e718-7e55-46c9-9827-46cd3f5aba62 admin admin] PATH_INFO: `/v3/users/f1e0e66beeab4be99ad4fd13a09afda6` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:06.195479 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f9e718-7e55-46c9-9827-46cd3f5aba62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1e0e66beeab4be99ad4fd13a09afda6)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:06.197626 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f9e718-7e55-46c9-9827-46cd3f5aba62 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:06.250653 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-9221820c-b4b0-46d3-b30b-0fdc3d0170f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7777652a2360413daea7c0a5eeb10d02'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:06.255461 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-f7f9e718-7e55-46c9-9827-46cd3f5aba62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1e0e66beeab4be99ad4fd13a09afda6'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:06.272933 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-9221820c-b4b0-46d3-b30b-0fdc3d0170f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7777652a2360413daea7c0a5eeb10d02'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:06.277753 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-f7f9e718-7e55-46c9-9827-46cd3f5aba62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1e0e66beeab4be99ad4fd13a09afda6'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:06.282053 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-9221820c-b4b0-46d3-b30b-0fdc3d0170f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7777652a2360413daea7c0a5eeb10d02'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:06.291497 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-f7f9e718-7e55-46c9-9827-46cd3f5aba62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1e0e66beeab4be99ad4fd13a09afda6'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:06.294755 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-9221820c-b4b0-46d3-b30b-0fdc3d0170f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7777652a2360413daea7c0a5eeb10d02'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:06.295707 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-9221820c-b4b0-46d3-b30b-0fdc3d0170f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7777652a2360413daea7c0a5eeb10d02'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:06.302163 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-f7f9e718-7e55-46c9-9827-46cd3f5aba62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1e0e66beeab4be99ad4fd13a09afda6'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:06.303180 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-f7f9e718-7e55-46c9-9827-46cd3f5aba62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1e0e66beeab4be99ad4fd13a09afda6'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:06.307082 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1059/2094] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:58:06 2022] DELETE /identity/v3/users/7777652a2360413daea7c0a5eeb10d02 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:06.311461 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d6c118-062d-4a94-b5e6-fd5a1b581db7 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:06.311773 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d6c118-062d-4a94-b5e6-fd5a1b581db7 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:06.312026 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d6c118-062d-4a94-b5e6-fd5a1b581db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:06.312989 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1036/2095] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:58:06 2022] DELETE /identity/v3/users/f1e0e66beeab4be99ad4fd13a09afda6 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:06.317346 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123ca79-624a-405d-b146-04766521e09c None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:06.317716 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123ca79-624a-405d-b146-04766521e09c None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:06.318061 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123ca79-624a-405d-b146-04766521e09c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:06.330457 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-47d6c118-062d-4a94-b5e6-fd5a1b581db7 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:06.331752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-47d6c118-062d-4a94-b5e6-fd5a1b581db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:06.341224 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-5123ca79-624a-405d-b146-04766521e09c None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:06.342357 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-5123ca79-624a-405d-b146-04766521e09c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:06.392377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1060/2096] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:06.402552 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d6c118-062d-4a94-b5e6-fd5a1b581db7 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:06.403975 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-47d6c118-062d-4a94-b5e6-fd5a1b581db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:06.415044 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1037/2097] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 01:58:06.426866 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5123ca79-624a-405d-b146-04766521e09c None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:06.428354 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-5123ca79-624a-405d-b146-04766521e09c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:06.428483 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:06.468636 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d6c118-062d-4a94-b5e6-fd5a1b581db7 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:06.472785 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9129e29e-4bd0-4e8c-82a4-423f3613d7a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:06.473563 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129e29e-4bd0-4e8c-82a4-423f3613d7a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:06.473916 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129e29e-4bd0-4e8c-82a4-423f3613d7a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:06.474264 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129e29e-4bd0-4e8c-82a4-423f3613d7a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:06.484967 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9129e29e-4bd0-4e8c-82a4-423f3613d7a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:06.486517 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9129e29e-4bd0-4e8c-82a4-423f3613d7a5 service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:06.494998 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5123ca79-624a-405d-b146-04766521e09c None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:06.496532 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1061/2098] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:58:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 01:58:06.499024 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6061d0-7815-4481-97cd-c5b0e81af28c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:06.499898 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6061d0-7815-4481-97cd-c5b0e81af28c service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:06.500339 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6061d0-7815-4481-97cd-c5b0e81af28c service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:06.500762 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6061d0-7815-4481-97cd-c5b0e81af28c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:06.511334 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6061d0-7815-4481-97cd-c5b0e81af28c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:06.513181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6061d0-7815-4481-97cd-c5b0e81af28c service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:06.523695 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1038/2099] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:58:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 01:58:06.973395 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9129e29e-4bd0-4e8c-82a4-423f3613d7a5 service neutron] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:06.976341 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-9129e29e-4bd0-4e8c-82a4-423f3613d7a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:07.001000 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6061d0-7815-4481-97cd-c5b0e81af28c service neutron] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:07.005356 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9129e29e-4bd0-4e8c-82a4-423f3613d7a5 service neutron] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:07.006149 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-6c6061d0-7815-4481-97cd-c5b0e81af28c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:07.011384 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a3395-4480-49e4-a2bc-b139947aed9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:07.012270 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a3395-4480-49e4-a2bc-b139947aed9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:07.012552 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a3395-4480-49e4-a2bc-b139947aed9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:07.012823 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a3395-4480-49e4-a2bc-b139947aed9f admin admin] PATH_INFO: `/v3/projects/9a8f1ba000cf4d62a1be41232bf73eaa` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:07.014141 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668a3395-4480-49e4-a2bc-b139947aed9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a8f1ba000cf4d62a1be41232bf73eaa)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:07.025877 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6061d0-7815-4481-97cd-c5b0e81af28c service neutron] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:07.031184 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668a3395-4480-49e4-a2bc-b139947aed9f admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:07.031780 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213da92b-d058-4f44-bd8b-31ac115d370f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:07.032686 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213da92b-d058-4f44-bd8b-31ac115d370f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:07.032923 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213da92b-d058-4f44-bd8b-31ac115d370f admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:07.033195 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213da92b-d058-4f44-bd8b-31ac115d370f admin admin] PATH_INFO: `/v3/projects/9951998e252845ca8927cf3432682075` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:07.034769 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213da92b-d058-4f44-bd8b-31ac115d370f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9951998e252845ca8927cf3432682075)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:07.036853 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213da92b-d058-4f44-bd8b-31ac115d370f admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:07.051921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:07.097624 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-213da92b-d058-4f44-bd8b-31ac115d370f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9951998e252845ca8927cf3432682075', 'request_id': 'req-213da92b-d058-4f44-bd8b-31ac115d370f'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:07.105332 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-213da92b-d058-4f44-bd8b-31ac115d370f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9951998e252845ca8927cf3432682075', 'request_id': 'req-213da92b-d058-4f44-bd8b-31ac115d370f'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:07.110606 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-668a3395-4480-49e4-a2bc-b139947aed9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a8f1ba000cf4d62a1be41232bf73eaa', 'request_id': 'req-668a3395-4480-49e4-a2bc-b139947aed9f'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:07.116824 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-213da92b-d058-4f44-bd8b-31ac115d370f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9951998e252845ca8927cf3432682075', 'request_id': 'req-213da92b-d058-4f44-bd8b-31ac115d370f'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:07.121018 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-668a3395-4480-49e4-a2bc-b139947aed9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a8f1ba000cf4d62a1be41232bf73eaa', 'request_id': 'req-668a3395-4480-49e4-a2bc-b139947aed9f'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:07.132057 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-213da92b-d058-4f44-bd8b-31ac115d370f admin admin] The token cache is being invalidate because project 9951998e252845ca8927cf3432682075 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:58:07.132641 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-213da92b-d058-4f44-bd8b-31ac115d370f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:07.133401 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-213da92b-d058-4f44-bd8b-31ac115d370f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:07.135244 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-668a3395-4480-49e4-a2bc-b139947aed9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a8f1ba000cf4d62a1be41232bf73eaa', 'request_id': 'req-668a3395-4480-49e4-a2bc-b139947aed9f'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:07.141729 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1039/2100] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:58:06 2022] DELETE /identity/v3/projects/9951998e252845ca8927cf3432682075 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 01:58:07.142736 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213da92b-d058-4f44-bd8b-31ac115d370f admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:07.144444 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-213da92b-d058-4f44-bd8b-31ac115d370f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:07.148052 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-668a3395-4480-49e4-a2bc-b139947aed9f admin admin] The token cache is being invalidate because project 9a8f1ba000cf4d62a1be41232bf73eaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:58:07.148348 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-668a3395-4480-49e4-a2bc-b139947aed9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:07.149015 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-668a3395-4480-49e4-a2bc-b139947aed9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:07.153424 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1062/2101] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:58:06 2022] DELETE /identity/v3/projects/9a8f1ba000cf4d62a1be41232bf73eaa => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:07.156351 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a3395-4480-49e4-a2bc-b139947aed9f admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:07.157707 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-668a3395-4480-49e4-a2bc-b139947aed9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:07.240091 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213da92b-d058-4f44-bd8b-31ac115d370f admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:07.257373 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a3395-4480-49e4-a2bc-b139947aed9f admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:07.273567 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dbadbb-37d2-42e1-81f6-59075e821141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:07.274576 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbadbb-37d2-42e1-81f6-59075e821141 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:07.274928 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbadbb-37d2-42e1-81f6-59075e821141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:07.275436 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbadbb-37d2-42e1-81f6-59075e821141 admin admin] PATH_INFO: `/v3/users/50f030020aba4f45b61582387495de6c` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:07.277148 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dbadbb-37d2-42e1-81f6-59075e821141 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50f030020aba4f45b61582387495de6c)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:07.279424 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dbadbb-37d2-42e1-81f6-59075e821141 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:07.332028 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3890546e-6497-43c6-a456-26f7eef581b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:07.332928 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3890546e-6497-43c6-a456-26f7eef581b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:07.333194 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3890546e-6497-43c6-a456-26f7eef581b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:07.333518 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3890546e-6497-43c6-a456-26f7eef581b1 admin admin] PATH_INFO: `/v3/users/0221b59c568c4c8aabb9a339df7f95d2` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:07.334917 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3890546e-6497-43c6-a456-26f7eef581b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0221b59c568c4c8aabb9a339df7f95d2)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:07.336965 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3890546e-6497-43c6-a456-26f7eef581b1 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:07.339733 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-47dbadbb-37d2-42e1-81f6-59075e821141 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50f030020aba4f45b61582387495de6c'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:07.342583 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:07.362528 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-47dbadbb-37d2-42e1-81f6-59075e821141 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50f030020aba4f45b61582387495de6c'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:07.372695 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-47dbadbb-37d2-42e1-81f6-59075e821141 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50f030020aba4f45b61582387495de6c'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:07.382483 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-47dbadbb-37d2-42e1-81f6-59075e821141 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50f030020aba4f45b61582387495de6c'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:07.383033 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-47dbadbb-37d2-42e1-81f6-59075e821141 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50f030020aba4f45b61582387495de6c'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:07.395293 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1063/2102] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:58:07 2022] DELETE /identity/v3/users/50f030020aba4f45b61582387495de6c => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:07.402268 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-3890546e-6497-43c6-a456-26f7eef581b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0221b59c568c4c8aabb9a339df7f95d2'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:07.404577 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:07.428133 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-3890546e-6497-43c6-a456-26f7eef581b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0221b59c568c4c8aabb9a339df7f95d2'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:07.437991 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-3890546e-6497-43c6-a456-26f7eef581b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0221b59c568c4c8aabb9a339df7f95d2'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:07.447855 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-3890546e-6497-43c6-a456-26f7eef581b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0221b59c568c4c8aabb9a339df7f95d2'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:07.449240 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-3890546e-6497-43c6-a456-26f7eef581b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0221b59c568c4c8aabb9a339df7f95d2'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:07.461592 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1040/2103] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:58:07 2022] DELETE /identity/v3/users/0221b59c568c4c8aabb9a339df7f95d2 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:07.815386 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dbadbb-37d2-42e1-81f6-59075e821141 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:07.817301 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-47dbadbb-37d2-42e1-81f6-59075e821141 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:07.890207 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dbadbb-37d2-42e1-81f6-59075e821141 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:07.911287 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3890546e-6497-43c6-a456-26f7eef581b1 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:07.913034 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-3890546e-6497-43c6-a456-26f7eef581b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:07.964296 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9220bb-a40f-4309-be82-cfcced2384f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:07.966241 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9220bb-a40f-4309-be82-cfcced2384f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:07.966526 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9220bb-a40f-4309-be82-cfcced2384f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:07.966805 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9220bb-a40f-4309-be82-cfcced2384f6 admin admin] PATH_INFO: `/v3/projects/5afdea1c69824282b3d48fd999d77c5c` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:07.968350 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9220bb-a40f-4309-be82-cfcced2384f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5afdea1c69824282b3d48fd999d77c5c)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:07.970904 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9220bb-a40f-4309-be82-cfcced2384f6 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:07.999128 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3890546e-6497-43c6-a456-26f7eef581b1 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:08.003965 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9d3889-59ea-4338-b67d-bbc67d5c5770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:08.004805 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d3889-59ea-4338-b67d-bbc67d5c5770 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:08.005083 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d3889-59ea-4338-b67d-bbc67d5c5770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:08.005360 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d3889-59ea-4338-b67d-bbc67d5c5770 admin admin] PATH_INFO: `/v3/projects/be90eb0787ce44bf923c70b10b72c01f` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:08.006581 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9d3889-59ea-4338-b67d-bbc67d5c5770 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be90eb0787ce44bf923c70b10b72c01f)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:08.008620 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9d3889-59ea-4338-b67d-bbc67d5c5770 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:08.027496 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-eb9220bb-a40f-4309-be82-cfcced2384f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5afdea1c69824282b3d48fd999d77c5c', 'request_id': 'req-eb9220bb-a40f-4309-be82-cfcced2384f6'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:08.034744 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-eb9220bb-a40f-4309-be82-cfcced2384f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5afdea1c69824282b3d48fd999d77c5c', 'request_id': 'req-eb9220bb-a40f-4309-be82-cfcced2384f6'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:08.044279 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-eb9220bb-a40f-4309-be82-cfcced2384f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5afdea1c69824282b3d48fd999d77c5c', 'request_id': 'req-eb9220bb-a40f-4309-be82-cfcced2384f6'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:08.056684 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-eb9220bb-a40f-4309-be82-cfcced2384f6 admin admin] The token cache is being invalidate because project 5afdea1c69824282b3d48fd999d77c5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:58:08.057082 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-eb9220bb-a40f-4309-be82-cfcced2384f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:08.057780 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-eb9220bb-a40f-4309-be82-cfcced2384f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:08.059468 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1064/2104] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:58:07 2022] DELETE /identity/v3/projects/5afdea1c69824282b3d48fd999d77c5c => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:08.064385 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97d2ce-658c-42e9-897a-e7040402b9e7 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:08.064742 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97d2ce-658c-42e9-897a-e7040402b9e7 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:08.065084 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97d2ce-658c-42e9-897a-e7040402b9e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:08.065376 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-3a9d3889-59ea-4338-b67d-bbc67d5c5770 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be90eb0787ce44bf923c70b10b72c01f', 'request_id': 'req-3a9d3889-59ea-4338-b67d-bbc67d5c5770'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:08.071738 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:08.075475 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-3a9d3889-59ea-4338-b67d-bbc67d5c5770 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be90eb0787ce44bf923c70b10b72c01f', 'request_id': 'req-3a9d3889-59ea-4338-b67d-bbc67d5c5770'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:08.084799 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-3a9d3889-59ea-4338-b67d-bbc67d5c5770 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be90eb0787ce44bf923c70b10b72c01f', 'request_id': 'req-3a9d3889-59ea-4338-b67d-bbc67d5c5770'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:08.091121 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-ac97d2ce-658c-42e9-897a-e7040402b9e7 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:08.093191 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-ac97d2ce-658c-42e9-897a-e7040402b9e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:08.096551 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-3a9d3889-59ea-4338-b67d-bbc67d5c5770 admin admin] The token cache is being invalidate because project be90eb0787ce44bf923c70b10b72c01f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:58:08.096846 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-3a9d3889-59ea-4338-b67d-bbc67d5c5770 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:08.097560 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-3a9d3889-59ea-4338-b67d-bbc67d5c5770 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:08.102488 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1041/2105] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:58:07 2022] DELETE /identity/v3/projects/be90eb0787ce44bf923c70b10b72c01f => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 01:58:08.103740 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9d3889-59ea-4338-b67d-bbc67d5c5770 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:08.105366 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-3a9d3889-59ea-4338-b67d-bbc67d5c5770 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:08.157459 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1065/2106] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:08.164875 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d4adb-b4f2-40fe-abd1-7270c1c2447d None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:08.165164 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d4adb-b4f2-40fe-abd1-7270c1c2447d None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:08.165442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d4adb-b4f2-40fe-abd1-7270c1c2447d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:08.181529 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:08.189019 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-962d4adb-b4f2-40fe-abd1-7270c1c2447d None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:08.189580 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9d3889-59ea-4338-b67d-bbc67d5c5770 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:08.191700 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-962d4adb-b4f2-40fe-abd1-7270c1c2447d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:08.197856 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9148ff-7747-4251-854b-868386831c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:08.198877 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9148ff-7747-4251-854b-868386831c58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:08.199293 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9148ff-7747-4251-854b-868386831c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:08.199771 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9148ff-7747-4251-854b-868386831c58 admin admin] PATH_INFO: `/v3/users/41fb9db1b90f471ba60a8897546ded68` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:08.213858 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9148ff-7747-4251-854b-868386831c58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41fb9db1b90f471ba60a8897546ded68)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:08.220244 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1066/2107] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 01:58:08.222045 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9148ff-7747-4251-854b-868386831c58 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:08.224608 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962d4adb-b4f2-40fe-abd1-7270c1c2447d None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:08.241891 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962d4adb-b4f2-40fe-abd1-7270c1c2447d None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:08.247634 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1652fee-c7a3-4f41-a882-7e1423edca27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:08.247880 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1652fee-c7a3-4f41-a882-7e1423edca27 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:08.248156 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1652fee-c7a3-4f41-a882-7e1423edca27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:08.248430 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1652fee-c7a3-4f41-a882-7e1423edca27 admin admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:08.249224 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1652fee-c7a3-4f41-a882-7e1423edca27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:58:08.249914 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1652fee-c7a3-4f41-a882-7e1423edca27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:08.251851 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1652fee-c7a3-4f41-a882-7e1423edca27 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:08.268766 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1067/2108] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 01:58:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:58:08.274586 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1652fee-c7a3-4f41-a882-7e1423edca27 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:08.296707 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1652fee-c7a3-4f41-a882-7e1423edca27 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:08.310500 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858f39d3-94b6-49e2-b420-1b56c071cfa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:08.311542 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f39d3-94b6-49e2-b420-1b56c071cfa0 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:08.312015 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f39d3-94b6-49e2-b420-1b56c071cfa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:08.314562 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f39d3-94b6-49e2-b420-1b56c071cfa0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:08.317990 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858f39d3-94b6-49e2-b420-1b56c071cfa0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:08.321467 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858f39d3-94b6-49e2-b420-1b56c071cfa0 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:08.337927 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-5f9148ff-7747-4251-854b-868386831c58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41fb9db1b90f471ba60a8897546ded68'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:08.359816 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-5f9148ff-7747-4251-854b-868386831c58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41fb9db1b90f471ba60a8897546ded68'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:08.368323 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1068/2109] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:58:08 2022] POST /identity/v3/projects => generated 387 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 01:58:08.373656 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858f39d3-94b6-49e2-b420-1b56c071cfa0 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:08.375400 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-5f9148ff-7747-4251-854b-868386831c58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41fb9db1b90f471ba60a8897546ded68'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:08.412992 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-5f9148ff-7747-4251-854b-868386831c58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41fb9db1b90f471ba60a8897546ded68'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:08.415135 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-5f9148ff-7747-4251-854b-868386831c58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41fb9db1b90f471ba60a8897546ded68'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:08.428637 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1042/2110] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:58:08 2022] DELETE /identity/v3/users/41fb9db1b90f471ba60a8897546ded68 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:08.477441 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858f39d3-94b6-49e2-b420-1b56c071cfa0 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:08.599897 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a83bbd-30d5-4033-ac5f-a2239b724f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:08.602294 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a83bbd-30d5-4033-ac5f-a2239b724f34 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:08.602740 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a83bbd-30d5-4033-ac5f-a2239b724f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:08.603823 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a83bbd-30d5-4033-ac5f-a2239b724f34 admin admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:08.606666 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a83bbd-30d5-4033-ac5f-a2239b724f34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:08.611787 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a83bbd-30d5-4033-ac5f-a2239b724f34 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:08.644781 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:08.657161 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1069/2111] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:58:08 2022] POST /identity/v3/users => generated 294 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:58:08.661122 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9148ff-7747-4251-854b-868386831c58 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:08.662580 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-5f9148ff-7747-4251-854b-868386831c58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:08.677256 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9148ff-7747-4251-854b-868386831c58 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:08.682462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008f29de-21f9-40f4-935c-d1623f42a300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:08.683163 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f29de-21f9-40f4-935c-d1623f42a300 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:08.683467 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f29de-21f9-40f4-935c-d1623f42a300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:08.683739 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f29de-21f9-40f4-935c-d1623f42a300 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:08.684800 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008f29de-21f9-40f4-935c-d1623f42a300 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:08.686597 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008f29de-21f9-40f4-935c-d1623f42a300 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:08.698012 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1043/2112] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:58:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:58:08.702392 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a83bbd-30d5-4033-ac5f-a2239b724f34 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:08.716446 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a83bbd-30d5-4033-ac5f-a2239b724f34 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:08.726007 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b7b674-52db-44ef-bfcf-4bd0151e55d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:08.726687 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b7b674-52db-44ef-bfcf-4bd0151e55d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:08.726969 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b7b674-52db-44ef-bfcf-4bd0151e55d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:08.727327 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b7b674-52db-44ef-bfcf-4bd0151e55d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:08.728135 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b7b674-52db-44ef-bfcf-4bd0151e55d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:08.729768 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b7b674-52db-44ef-bfcf-4bd0151e55d6 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:08.749688 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1070/2113] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:58:08 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:58:08.754097 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008f29de-21f9-40f4-935c-d1623f42a300 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:08.768668 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008f29de-21f9-40f4-935c-d1623f42a300 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:08.777260 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7931a5d-1cc5-4418-88c3-a353e077e12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:08.778907 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7931a5d-1cc5-4418-88c3-a353e077e12a admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:08.779194 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7931a5d-1cc5-4418-88c3-a353e077e12a admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:08.780025 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7931a5d-1cc5-4418-88c3-a353e077e12a admin admin] PATH_INFO: `/v3/projects/f5a9a97b5ff54aea80de0ad344b8871a/users/e856fc36a790404fa05e97eeddd7b30e/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:08.804562 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7931a5d-1cc5-4418-88c3-a353e077e12a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5a9a97b5ff54aea80de0ad344b8871a, user_id=e856fc36a790404fa05e97eeddd7b30e, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:08.807707 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7931a5d-1cc5-4418-88c3-a353e077e12a admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:08.836247 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1044/2114] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:58:08 2022] PUT /identity/v3/projects/f5a9a97b5ff54aea80de0ad344b8871a/users/e856fc36a790404fa05e97eeddd7b30e/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:08.858301 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7931a5d-1cc5-4418-88c3-a353e077e12a admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:08.859030 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8dd6e-6de1-4427-8d6f-c45b5eb79a70 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:08.860310 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-c7931a5d-1cc5-4418-88c3-a353e077e12a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:08.860799 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8dd6e-6de1-4427-8d6f-c45b5eb79a70 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:08.861221 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8dd6e-6de1-4427-8d6f-c45b5eb79a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:08.915419 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-0ed8dd6e-6de1-4427-8d6f-c45b5eb79a70 None None] MFA Rules not processed for user `e856fc36a790404fa05e97eeddd7b30e`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:08.916945 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-0ed8dd6e-6de1-4427-8d6f-c45b5eb79a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:08.957989 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7931a5d-1cc5-4418-88c3-a353e077e12a admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:08.963391 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098574e5-ca75-49e4-a4ef-88c63480b7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:08.964355 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098574e5-ca75-49e4-a4ef-88c63480b7ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:08.964641 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098574e5-ca75-49e4-a4ef-88c63480b7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:08.964969 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098574e5-ca75-49e4-a4ef-88c63480b7ad admin admin] PATH_INFO: `/v3/projects/a6dc5009e9e24dbc8b8cea2436e35f5d` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:08.966202 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098574e5-ca75-49e4-a4ef-88c63480b7ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6dc5009e9e24dbc8b8cea2436e35f5d)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:08.968240 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098574e5-ca75-49e4-a4ef-88c63480b7ad admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:09.012936 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1071/2115] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:09.017571 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed8dd6e-6de1-4427-8d6f-c45b5eb79a70 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:09.029305 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed8dd6e-6de1-4427-8d6f-c45b5eb79a70 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:09.033020 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:09.042596 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-098574e5-ca75-49e4-a4ef-88c63480b7ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6dc5009e9e24dbc8b8cea2436e35f5d', 'request_id': 'req-098574e5-ca75-49e4-a4ef-88c63480b7ad'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:09.050457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-098574e5-ca75-49e4-a4ef-88c63480b7ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6dc5009e9e24dbc8b8cea2436e35f5d', 'request_id': 'req-098574e5-ca75-49e4-a4ef-88c63480b7ad'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:09.060027 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-098574e5-ca75-49e4-a4ef-88c63480b7ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6dc5009e9e24dbc8b8cea2436e35f5d', 'request_id': 'req-098574e5-ca75-49e4-a4ef-88c63480b7ad'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:09.074598 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-098574e5-ca75-49e4-a4ef-88c63480b7ad admin admin] The token cache is being invalidate because project a6dc5009e9e24dbc8b8cea2436e35f5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:58:09.074891 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-098574e5-ca75-49e4-a4ef-88c63480b7ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:09.075561 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-098574e5-ca75-49e4-a4ef-88c63480b7ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:09.077781 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1045/2116] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:58:08 2022] DELETE /identity/v3/projects/a6dc5009e9e24dbc8b8cea2436e35f5d => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 01:58:09.082797 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516f19f-f88a-44f3-838d-983bcdc06037 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:09.083073 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516f19f-f88a-44f3-838d-983bcdc06037 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:09.083386 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516f19f-f88a-44f3-838d-983bcdc06037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:09.098818 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe2a0e9-1bd4-4eb6-80fa-f938a5f35dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:09.099827 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2a0e9-1bd4-4eb6-80fa-f938a5f35dc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:09.100128 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2a0e9-1bd4-4eb6-80fa-f938a5f35dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:09.100495 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2a0e9-1bd4-4eb6-80fa-f938a5f35dc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:09.101367 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe2a0e9-1bd4-4eb6-80fa-f938a5f35dc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:09.103053 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-1516f19f-f88a-44f3-838d-983bcdc06037 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:09.104110 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe2a0e9-1bd4-4eb6-80fa-f938a5f35dc5 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:09.104336 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-1516f19f-f88a-44f3-838d-983bcdc06037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:09.119523 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1046/2117] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 01:58:09.121538 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796212d2-527d-47ac-879e-41d4d28430a2 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:09.121813 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796212d2-527d-47ac-879e-41d4d28430a2 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:09.122088 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796212d2-527d-47ac-879e-41d4d28430a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:09.137873 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1072/2118] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:58:09 2022] POST /identity/v3/projects => generated 387 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 01:58:09.140033 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe2a0e9-1bd4-4eb6-80fa-f938a5f35dc5 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:09.144076 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-ebe2a0e9-1bd4-4eb6-80fa-f938a5f35dc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:09.144744 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-796212d2-527d-47ac-879e-41d4d28430a2 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:09.145912 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-796212d2-527d-47ac-879e-41d4d28430a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:09.208940 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1047/2119] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:09.211632 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe2a0e9-1bd4-4eb6-80fa-f938a5f35dc5 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:09.213164 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796212d2-527d-47ac-879e-41d4d28430a2 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:09.216574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dee3010-ee75-4888-b243-6bdef1a956e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:09.217234 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee3010-ee75-4888-b243-6bdef1a956e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:09.217514 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee3010-ee75-4888-b243-6bdef1a956e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:09.217913 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee3010-ee75-4888-b243-6bdef1a956e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:09.218477 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dee3010-ee75-4888-b243-6bdef1a956e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:09.220426 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dee3010-ee75-4888-b243-6bdef1a956e5 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:09.223385 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796212d2-527d-47ac-879e-41d4d28430a2 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:09.228624 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6966b00-ed73-4032-8ed8-5c79194ef9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:09.229197 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6966b00-ed73-4032-8ed8-5c79194ef9f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:09.229548 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6966b00-ed73-4032-8ed8-5c79194ef9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:09.230960 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6966b00-ed73-4032-8ed8-5c79194ef9f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:09.230960 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6966b00-ed73-4032-8ed8-5c79194ef9f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:58:09.231154 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6966b00-ed73-4032-8ed8-5c79194ef9f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:09.235498 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6966b00-ed73-4032-8ed8-5c79194ef9f2 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:09.236197 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:09.250911 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1048/2120] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 01:58:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:58:09.252330 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6966b00-ed73-4032-8ed8-5c79194ef9f2 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:09.268300 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1073/2121] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:58:09 2022] POST /identity/v3/users => generated 294 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:58:09.272154 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dee3010-ee75-4888-b243-6bdef1a956e5 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:09.272372 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6966b00-ed73-4032-8ed8-5c79194ef9f2 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:09.277622 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe9c546-1c6d-4cfe-9f72-238760e32386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:09.278294 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9c546-1c6d-4cfe-9f72-238760e32386 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:09.278576 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9c546-1c6d-4cfe-9f72-238760e32386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:09.279139 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9c546-1c6d-4cfe-9f72-238760e32386 admin admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:09.279908 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe9c546-1c6d-4cfe-9f72-238760e32386 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:09.281812 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe9c546-1c6d-4cfe-9f72-238760e32386 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:09.282987 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dee3010-ee75-4888-b243-6bdef1a956e5 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:09.298588 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c1a00b-4676-43f2-bcbe-93bd3b6186de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:09.299731 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1a00b-4676-43f2-bcbe-93bd3b6186de admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:09.300253 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1a00b-4676-43f2-bcbe-93bd3b6186de admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:09.301129 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1a00b-4676-43f2-bcbe-93bd3b6186de admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:09.302528 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c1a00b-4676-43f2-bcbe-93bd3b6186de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:09.305323 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c1a00b-4676-43f2-bcbe-93bd3b6186de admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:09.324774 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1049/2122] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:58:09 2022] POST /identity/v3/projects => generated 383 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:58:09.326307 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe9c546-1c6d-4cfe-9f72-238760e32386 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:09.326558 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1074/2123] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 01:58:09.330261 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c1a00b-4676-43f2-bcbe-93bd3b6186de admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:09.348566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c1a00b-4676-43f2-bcbe-93bd3b6186de admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:09.452374 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe9c546-1c6d-4cfe-9f72-238760e32386 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:09.460890 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564349e-fcee-4ea2-abcd-1a7b79eccd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:09.461509 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564349e-fcee-4ea2-abcd-1a7b79eccd0e admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:09.461792 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564349e-fcee-4ea2-abcd-1a7b79eccd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:09.462063 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564349e-fcee-4ea2-abcd-1a7b79eccd0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:09.462803 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3564349e-fcee-4ea2-abcd-1a7b79eccd0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:09.464487 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3564349e-fcee-4ea2-abcd-1a7b79eccd0e admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:09.465628 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f74cd3f-d1ba-439a-bb8d-fa9f283700fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:09.466589 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f74cd3f-d1ba-439a-bb8d-fa9f283700fc admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:09.467242 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f74cd3f-d1ba-439a-bb8d-fa9f283700fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:09.467546 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f74cd3f-d1ba-439a-bb8d-fa9f283700fc admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:09.468262 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f74cd3f-d1ba-439a-bb8d-fa9f283700fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:09.470946 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f74cd3f-d1ba-439a-bb8d-fa9f283700fc admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:09.479160 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1075/2124] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:58:09.481526 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564349e-fcee-4ea2-abcd-1a7b79eccd0e admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:09.500434 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1050/2125] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:58:09 2022] POST /identity/v3/users => generated 292 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:58:09.503901 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f74cd3f-d1ba-439a-bb8d-fa9f283700fc admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:09.515833 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564349e-fcee-4ea2-abcd-1a7b79eccd0e admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:09.516185 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f74cd3f-d1ba-439a-bb8d-fa9f283700fc admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:09.521620 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c5546-f0f1-417a-930e-70105a33f8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:09.522354 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c5546-f0f1-417a-930e-70105a33f8aa admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:09.522833 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c5546-f0f1-417a-930e-70105a33f8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:09.523057 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c5546-f0f1-417a-930e-70105a33f8aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:09.523852 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6c5546-f0f1-417a-930e-70105a33f8aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:09.527402 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccaecc6-cbec-4eab-b4aa-6d75eddc215b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:09.527577 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6c5546-f0f1-417a-930e-70105a33f8aa admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:09.527675 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccaecc6-cbec-4eab-b4aa-6d75eddc215b admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:09.527675 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccaecc6-cbec-4eab-b4aa-6d75eddc215b admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:09.527675 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccaecc6-cbec-4eab-b4aa-6d75eddc215b admin admin] PATH_INFO: `/v3/projects/0db66ad3f2624bdba88f3aabedc7f3a4/users/3eda796074e74c8fad4a3a8e00c1605c/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:09.538906 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1051/2126] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 01:58:09.543384 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c5546-f0f1-417a-930e-70105a33f8aa admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:09.554957 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccaecc6-cbec-4eab-b4aa-6d75eddc215b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0db66ad3f2624bdba88f3aabedc7f3a4, user_id=3eda796074e74c8fad4a3a8e00c1605c, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:09.555449 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c5546-f0f1-417a-930e-70105a33f8aa admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:09.558530 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccaecc6-cbec-4eab-b4aa-6d75eddc215b admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:09.560475 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d09bde8-d6a3-4b22-828e-bdd1db963b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:09.561221 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d09bde8-d6a3-4b22-828e-bdd1db963b35 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:09.561574 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d09bde8-d6a3-4b22-828e-bdd1db963b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:09.562032 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d09bde8-d6a3-4b22-828e-bdd1db963b35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:09.562874 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d09bde8-d6a3-4b22-828e-bdd1db963b35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:09.565063 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d09bde8-d6a3-4b22-828e-bdd1db963b35 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:09.566832 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:09.586609 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1076/2127] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:58:09 2022] PUT /identity/v3/projects/0db66ad3f2624bdba88f3aabedc7f3a4/users/3eda796074e74c8fad4a3a8e00c1605c/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:09.589142 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1052/2128] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:58:09.596979 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb2bb8-a985-499a-82a3-02d7bf4ef55d None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:09.597278 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb2bb8-a985-499a-82a3-02d7bf4ef55d None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:09.597568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb2bb8-a985-499a-82a3-02d7bf4ef55d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:09.599451 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d09bde8-d6a3-4b22-828e-bdd1db963b35 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:09.620016 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d09bde8-d6a3-4b22-828e-bdd1db963b35 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:09.648861 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-aacb2bb8-a985-499a-82a3-02d7bf4ef55d None None] MFA Rules not processed for user `3eda796074e74c8fad4a3a8e00c1605c`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:09.650211 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-aacb2bb8-a985-499a-82a3-02d7bf4ef55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:09.730864 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0dee9b-957f-42b4-b2b4-8df1aa2ef1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:09.731847 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0dee9b-957f-42b4-b2b4-8df1aa2ef1e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:09.732213 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0dee9b-957f-42b4-b2b4-8df1aa2ef1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:09.732714 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0dee9b-957f-42b4-b2b4-8df1aa2ef1e3 admin admin] PATH_INFO: `/v3/projects/5a7db67320b04863ac7d28369425dd19/users/3d72a03c4eec4412856ee48cb967c46d/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:09.750452 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1077/2129] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:09.751603 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb2bb8-a985-499a-82a3-02d7bf4ef55d None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:09.759489 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0dee9b-957f-42b4-b2b4-8df1aa2ef1e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a7db67320b04863ac7d28369425dd19, user_id=3d72a03c4eec4412856ee48cb967c46d, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:09.763977 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0dee9b-957f-42b4-b2b4-8df1aa2ef1e3 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:09.766433 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb2bb8-a985-499a-82a3-02d7bf4ef55d None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:09.776322 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864fa3c4-3c22-4376-884a-94ec0bb87c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:09.777040 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864fa3c4-3c22-4376-884a-94ec0bb87c85 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:09.777529 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864fa3c4-3c22-4376-884a-94ec0bb87c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:09.777926 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864fa3c4-3c22-4376-884a-94ec0bb87c85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:09.778698 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864fa3c4-3c22-4376-884a-94ec0bb87c85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:09.781218 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864fa3c4-3c22-4376-884a-94ec0bb87c85 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:09.789333 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1053/2130] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:58:09 2022] PUT /identity/v3/projects/5a7db67320b04863ac7d28369425dd19/users/3d72a03c4eec4412856ee48cb967c46d/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:09.798884 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8defecc0-2c67-4492-a18e-c0733209ec8b None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:09.798884 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8defecc0-2c67-4492-a18e-c0733209ec8b None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:09.799268 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8defecc0-2c67-4492-a18e-c0733209ec8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:09.820557 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-8defecc0-2c67-4492-a18e-c0733209ec8b None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:09.822350 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-8defecc0-2c67-4492-a18e-c0733209ec8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:09.857132 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1078/2131] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:58:09 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:58:09.858377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864fa3c4-3c22-4376-884a-94ec0bb87c85 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:09.895078 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:09.929277 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1054/2132] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:09.939731 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8defecc0-2c67-4492-a18e-c0733209ec8b None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:09.941330 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-8defecc0-2c67-4492-a18e-c0733209ec8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:09.979619 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864fa3c4-3c22-4376-884a-94ec0bb87c85 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:09.986306 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ca625a-b344-4847-ace4-b677e05a3850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:09.987360 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca625a-b344-4847-ace4-b677e05a3850 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:09.987699 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca625a-b344-4847-ace4-b677e05a3850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:09.987969 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca625a-b344-4847-ace4-b677e05a3850 admin admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:09.989243 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ca625a-b344-4847-ace4-b677e05a3850 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:09.991245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ca625a-b344-4847-ace4-b677e05a3850 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:10.025993 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1079/2133] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:58:09 2022] POST /identity/v3/users => generated 299 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:58:10.030576 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ca625a-b344-4847-ace4-b677e05a3850 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:10.045054 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8defecc0-2c67-4492-a18e-c0733209ec8b None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:10.053558 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ca625a-b344-4847-ace4-b677e05a3850 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:10.060878 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bd10fa-dccc-475f-8151-a3df9a6d0281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:10.061992 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bd10fa-dccc-475f-8151-a3df9a6d0281 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:10.062282 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bd10fa-dccc-475f-8151-a3df9a6d0281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:10.062564 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bd10fa-dccc-475f-8151-a3df9a6d0281 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:10.063348 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bd10fa-dccc-475f-8151-a3df9a6d0281 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:10.065185 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bd10fa-dccc-475f-8151-a3df9a6d0281 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:10.082893 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1080/2134] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:58:10 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 01:58:10.084183 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bd10fa-dccc-475f-8151-a3df9a6d0281 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:10.090301 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:10.104498 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bd10fa-dccc-475f-8151-a3df9a6d0281 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:10.112721 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb2148-578f-4835-b127-44477047698b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:10.113629 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb2148-578f-4835-b127-44477047698b admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:10.113912 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb2148-578f-4835-b127-44477047698b admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:10.114191 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb2148-578f-4835-b127-44477047698b admin admin] PATH_INFO: `/v3/projects/1adc257f84124778aee8a0ec1360c082/users/dae643f7f0564e65947c2b1d7d5167e4/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:10.118363 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db66fcd0-de5b-41b3-9835-788ed1e5d451 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:10.119871 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66fcd0-de5b-41b3-9835-788ed1e5d451 service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:10.120769 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66fcd0-de5b-41b3-9835-788ed1e5d451 service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:10.123155 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66fcd0-de5b-41b3-9835-788ed1e5d451 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:10.139198 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db66fcd0-de5b-41b3-9835-788ed1e5d451 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:10.141839 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cb2148-578f-4835-b127-44477047698b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1adc257f84124778aee8a0ec1360c082, user_id=dae643f7f0564e65947c2b1d7d5167e4, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:10.141973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db66fcd0-de5b-41b3-9835-788ed1e5d451 service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:10.143274 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cb2148-578f-4835-b127-44477047698b admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:10.165587 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1055/2135] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:58:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 01:58:10.179300 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1081/2136] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:58:10 2022] PUT /identity/v3/projects/1adc257f84124778aee8a0ec1360c082/users/dae643f7f0564e65947c2b1d7d5167e4/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 01:58:10.183614 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db66fcd0-de5b-41b3-9835-788ed1e5d451 service neutron] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:10.205187 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db66fcd0-de5b-41b3-9835-788ed1e5d451 service neutron] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:10.344425 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ad196-f178-4f2a-951f-b286b1d87372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:10.346289 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ad196-f178-4f2a-951f-b286b1d87372 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:10.354154 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ad196-f178-4f2a-951f-b286b1d87372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:10.354154 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ad196-f178-4f2a-951f-b286b1d87372 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:10.355076 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309ad196-f178-4f2a-951f-b286b1d87372 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:10.357396 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309ad196-f178-4f2a-951f-b286b1d87372 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:10.370491 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1056/2137] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:58:10 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 01:58:10.374961 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb2148-578f-4835-b127-44477047698b admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:10.387734 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb2148-578f-4835-b127-44477047698b admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:10.393179 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8062c7a-3a4a-440a-a1a9-6186d102e59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:10.394373 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8062c7a-3a4a-440a-a1a9-6186d102e59f admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:10.394923 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8062c7a-3a4a-440a-a1a9-6186d102e59f admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:10.395365 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8062c7a-3a4a-440a-a1a9-6186d102e59f admin admin] PATH_INFO: `/v3/projects/1adc257f84124778aee8a0ec1360c082/users/dae643f7f0564e65947c2b1d7d5167e4/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:10.397865 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8062c7a-3a4a-440a-a1a9-6186d102e59f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1adc257f84124778aee8a0ec1360c082, user_id=dae643f7f0564e65947c2b1d7d5167e4, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:10.400947 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8062c7a-3a4a-440a-a1a9-6186d102e59f admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:10.419344 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1082/2138] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:58:10 2022] PUT /identity/v3/projects/1adc257f84124778aee8a0ec1360c082/users/dae643f7f0564e65947c2b1d7d5167e4/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:10.432158 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e65f5d-1677-4908-86fe-093bc2cb8958 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:10.432552 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e65f5d-1677-4908-86fe-093bc2cb8958 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:10.432930 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e65f5d-1677-4908-86fe-093bc2cb8958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:10.473762 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-d5e65f5d-1677-4908-86fe-093bc2cb8958 None None] MFA Rules not processed for user `dae643f7f0564e65947c2b1d7d5167e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:10.475334 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-d5e65f5d-1677-4908-86fe-093bc2cb8958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:10.536172 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc74ef5-42d7-4f3a-abdd-cf42c1f99779 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:10.536790 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc74ef5-42d7-4f3a-abdd-cf42c1f99779 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:10.537269 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc74ef5-42d7-4f3a-abdd-cf42c1f99779 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:10.539506 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1083/2139] 10.222.0.26 () {54 vars in 921 bytes} [Thu Jun 23 01:58:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:58:10.548737 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a11e38-babf-4923-97df-7a73961a5665 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:10.549463 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a11e38-babf-4923-97df-7a73961a5665 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:10.550197 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a11e38-babf-4923-97df-7a73961a5665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:10.573524 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1057/2140] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:10 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:10.577910 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-c6a11e38-babf-4923-97df-7a73961a5665 None None] MFA Rules not processed for user `e5bf3baee1b843fa89b65e5485f6bebb`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:10.579419 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-c6a11e38-babf-4923-97df-7a73961a5665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:10.587844 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e65f5d-1677-4908-86fe-093bc2cb8958 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:10.589664 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-d5e65f5d-1677-4908-86fe-093bc2cb8958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:10.687328 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e65f5d-1677-4908-86fe-093bc2cb8958 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:10.695996 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1084/2141] 10.222.0.26 () {58 vars in 1018 bytes} [Thu Jun 23 01:58:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:10.697223 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec34e5-c376-43e9-8027-418914a9074c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:10.697459 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a11e38-babf-4923-97df-7a73961a5665 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:10.698031 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec34e5-c376-43e9-8027-418914a9074c service cinder] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:10.698321 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec34e5-c376-43e9-8027-418914a9074c service cinder] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:10.698577 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ec34e5-c376-43e9-8027-418914a9074c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:10.700012 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-c6a11e38-babf-4923-97df-7a73961a5665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:10.724653 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ec34e5-c376-43e9-8027-418914a9074c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:10.726956 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ec34e5-c376-43e9-8027-418914a9074c service cinder] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:10.750396 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1058/2142] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:58:10 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 01:58:10.752265 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec34e5-c376-43e9-8027-418914a9074c service cinder] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:10.754070 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-49ec34e5-c376-43e9-8027-418914a9074c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:10.769494 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:10.777296 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ec34e5-c376-43e9-8027-418914a9074c service cinder] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:10.782856 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8152ba-4c8f-4610-865a-9c5f1c0b59bb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5bf3baee1b843fa89b65e5485f6bebb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:10.783825 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8152ba-4c8f-4610-865a-9c5f1c0b59bb service placement] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:10.783825 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8152ba-4c8f-4610-865a-9c5f1c0b59bb service placement] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:10.784098 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8152ba-4c8f-4610-865a-9c5f1c0b59bb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:10.793608 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8152ba-4c8f-4610-865a-9c5f1c0b59bb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:10.795320 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8152ba-4c8f-4610-865a-9c5f1c0b59bb service placement] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:10.805995 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1059/2143] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:58:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:58:10.806811 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8152ba-4c8f-4610-865a-9c5f1c0b59bb service placement] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:10.809058 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-fb8152ba-4c8f-4610-865a-9c5f1c0b59bb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:10.837958 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:10.843114 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a11e38-babf-4923-97df-7a73961a5665 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:10.853214 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9738284e-fae0-4fa9-8833-b33ac505e211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:10.854222 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738284e-fae0-4fa9-8833-b33ac505e211 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:10.854676 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738284e-fae0-4fa9-8833-b33ac505e211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:10.855692 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738284e-fae0-4fa9-8833-b33ac505e211 admin admin] PATH_INFO: `/v3/users/56362ad5e50c4f4e8fb694f845d02924` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:10.856926 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9738284e-fae0-4fa9-8833-b33ac505e211 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56362ad5e50c4f4e8fb694f845d02924)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:10.859615 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9738284e-fae0-4fa9-8833-b33ac505e211 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:10.908499 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8152ba-4c8f-4610-865a-9c5f1c0b59bb service placement] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:10.914896 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528777e7-1d3e-4061-bcea-975bfda4b38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:10.916523 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528777e7-1d3e-4061-bcea-975bfda4b38e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:10.917102 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528777e7-1d3e-4061-bcea-975bfda4b38e admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:10.917459 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528777e7-1d3e-4061-bcea-975bfda4b38e admin admin] PATH_INFO: `/v3/users/96f434251eef46958f16259266ac5b33` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:10.919307 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528777e7-1d3e-4061-bcea-975bfda4b38e admin admin] RBAC: Authorizing `identity:delete_user(user_id=96f434251eef46958f16259266ac5b33)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:10.922229 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528777e7-1d3e-4061-bcea-975bfda4b38e admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:10.929019 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-9738284e-fae0-4fa9-8833-b33ac505e211 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56362ad5e50c4f4e8fb694f845d02924'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:10.951648 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-9738284e-fae0-4fa9-8833-b33ac505e211 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56362ad5e50c4f4e8fb694f845d02924'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:10.965699 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-9738284e-fae0-4fa9-8833-b33ac505e211 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56362ad5e50c4f4e8fb694f845d02924'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:10.976338 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-9738284e-fae0-4fa9-8833-b33ac505e211 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56362ad5e50c4f4e8fb694f845d02924'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:10.977263 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-9738284e-fae0-4fa9-8833-b33ac505e211 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56362ad5e50c4f4e8fb694f845d02924'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:10.986509 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1085/2144] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:58:10 2022] DELETE /identity/v3/users/56362ad5e50c4f4e8fb694f845d02924 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:11.014959 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-528777e7-1d3e-4061-bcea-975bfda4b38e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96f434251eef46958f16259266ac5b33'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:11.039357 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-528777e7-1d3e-4061-bcea-975bfda4b38e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96f434251eef46958f16259266ac5b33'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:11.053680 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-528777e7-1d3e-4061-bcea-975bfda4b38e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96f434251eef46958f16259266ac5b33'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:11.066299 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-528777e7-1d3e-4061-bcea-975bfda4b38e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96f434251eef46958f16259266ac5b33'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:11.067129 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-528777e7-1d3e-4061-bcea-975bfda4b38e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96f434251eef46958f16259266ac5b33'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:11.074407 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1060/2145] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:58:10 2022] DELETE /identity/v3/users/96f434251eef46958f16259266ac5b33 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:11.078848 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90fb3b4-7845-4179-a755-ae4de9d20f94 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:11.079401 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90fb3b4-7845-4179-a755-ae4de9d20f94 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:11.079643 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90fb3b4-7845-4179-a755-ae4de9d20f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:11.103691 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-b90fb3b4-7845-4179-a755-ae4de9d20f94 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:11.105928 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-b90fb3b4-7845-4179-a755-ae4de9d20f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:11.228661 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528777e7-1d3e-4061-bcea-975bfda4b38e admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:11.230303 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-528777e7-1d3e-4061-bcea-975bfda4b38e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:11.244381 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1086/2146] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:11.259427 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90fb3b4-7845-4179-a755-ae4de9d20f94 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:11.261621 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-b90fb3b4-7845-4179-a755-ae4de9d20f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:11.383057 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528777e7-1d3e-4061-bcea-975bfda4b38e admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:11.389031 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030a27d3-1c2e-4933-8f83-e561b509f677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:11.390055 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a27d3-1c2e-4933-8f83-e561b509f677 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:11.390340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a27d3-1c2e-4933-8f83-e561b509f677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:11.390625 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a27d3-1c2e-4933-8f83-e561b509f677 admin admin] PATH_INFO: `/v3/projects/c435aab218b8498585477269fe5ecc60` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:11.392215 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030a27d3-1c2e-4933-8f83-e561b509f677 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c435aab218b8498585477269fe5ecc60)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:11.394769 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030a27d3-1c2e-4933-8f83-e561b509f677 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:11.396859 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90fb3b4-7845-4179-a755-ae4de9d20f94 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:11.407086 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e911e4-3af7-4092-a77a-9b5a81b2f5d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:11.407618 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e911e4-3af7-4092-a77a-9b5a81b2f5d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:11.407759 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:11.407913 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e911e4-3af7-4092-a77a-9b5a81b2f5d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:11.408202 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e911e4-3af7-4092-a77a-9b5a81b2f5d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:11.427815 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e911e4-3af7-4092-a77a-9b5a81b2f5d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:11.430525 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e911e4-3af7-4092-a77a-9b5a81b2f5d2 service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:11.449286 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1087/2147] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:58:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:58:11.496404 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-030a27d3-1c2e-4933-8f83-e561b509f677 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c435aab218b8498585477269fe5ecc60', 'request_id': 'req-030a27d3-1c2e-4933-8f83-e561b509f677'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:11.509875 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-030a27d3-1c2e-4933-8f83-e561b509f677 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c435aab218b8498585477269fe5ecc60', 'request_id': 'req-030a27d3-1c2e-4933-8f83-e561b509f677'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:11.520948 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-030a27d3-1c2e-4933-8f83-e561b509f677 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c435aab218b8498585477269fe5ecc60', 'request_id': 'req-030a27d3-1c2e-4933-8f83-e561b509f677'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:11.541295 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-030a27d3-1c2e-4933-8f83-e561b509f677 admin admin] The token cache is being invalidate because project c435aab218b8498585477269fe5ecc60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:58:11.542450 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-030a27d3-1c2e-4933-8f83-e561b509f677 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:11.551751 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-030a27d3-1c2e-4933-8f83-e561b509f677 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:11.553510 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1061/2148] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:58:11 2022] DELETE /identity/v3/projects/c435aab218b8498585477269fe5ecc60 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:11.560091 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ed041-ae0f-4478-a049-34a9eed571f9 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:11.560396 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ed041-ae0f-4478-a049-34a9eed571f9 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:11.560675 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ed041-ae0f-4478-a049-34a9eed571f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:11.609432 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-1b7ed041-ae0f-4478-a049-34a9eed571f9 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:11.610677 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-1b7ed041-ae0f-4478-a049-34a9eed571f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:11.646292 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:11.674929 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030a27d3-1c2e-4933-8f83-e561b509f677 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:11.677268 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-030a27d3-1c2e-4933-8f83-e561b509f677 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:11.779721 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1088/2149] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:11.781421 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ecf0f2-ed24-48a2-8106-ef391d5fed46 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:11.781761 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ecf0f2-ed24-48a2-8106-ef391d5fed46 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:11.781976 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ecf0f2-ed24-48a2-8106-ef391d5fed46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:11.801556 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030a27d3-1c2e-4933-8f83-e561b509f677 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:11.805350 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-63ecf0f2-ed24-48a2-8106-ef391d5fed46 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:11.806720 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-63ecf0f2-ed24-48a2-8106-ef391d5fed46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:11.809945 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a897de67-6b0d-4210-b760-995f1f15774f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:11.810796 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a897de67-6b0d-4210-b760-995f1f15774f service cinder] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:11.811068 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a897de67-6b0d-4210-b760-995f1f15774f service cinder] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:11.811379 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a897de67-6b0d-4210-b760-995f1f15774f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:11.813576 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-a897de67-6b0d-4210-b760-995f1f15774f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:11.819687 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1089/2150] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:11.821157 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ecf0f2-ed24-48a2-8106-ef391d5fed46 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:11.835396 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ecf0f2-ed24-48a2-8106-ef391d5fed46 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:11.840752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229a5dbe-cb40-40b2-aebc-810fa632d677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:11.841545 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a5dbe-cb40-40b2-aebc-810fa632d677 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:11.841888 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a5dbe-cb40-40b2-aebc-810fa632d677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:11.842227 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a5dbe-cb40-40b2-aebc-810fa632d677 admin admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:11.843066 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229a5dbe-cb40-40b2-aebc-810fa632d677 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:58:11.843549 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229a5dbe-cb40-40b2-aebc-810fa632d677 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:11.851884 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229a5dbe-cb40-40b2-aebc-810fa632d677 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:11.870259 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1090/2151] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 01:58:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:58:11.871359 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229a5dbe-cb40-40b2-aebc-810fa632d677 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:11.884406 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229a5dbe-cb40-40b2-aebc-810fa632d677 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:11.898436 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c44eb6-f156-42ff-b70f-6c7a90562419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:11.900214 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c44eb6-f156-42ff-b70f-6c7a90562419 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:11.901409 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c44eb6-f156-42ff-b70f-6c7a90562419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:11.901559 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c44eb6-f156-42ff-b70f-6c7a90562419 admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:11.903985 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c44eb6-f156-42ff-b70f-6c7a90562419 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:11.906167 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c44eb6-f156-42ff-b70f-6c7a90562419 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:11.943476 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a897de67-6b0d-4210-b760-995f1f15774f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:11.945144 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a897de67-6b0d-4210-b760-995f1f15774f service cinder] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:11.960848 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1091/2152] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:58:11 2022] POST /identity/v3/projects => generated 375 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:58:11.961536 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c44eb6-f156-42ff-b70f-6c7a90562419 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:11.965684 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1062/2153] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:58:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:58:11.967083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-e4c44eb6-f156-42ff-b70f-6c7a90562419 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:11.969293 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a897de67-6b0d-4210-b760-995f1f15774f service cinder] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:11.999063 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a897de67-6b0d-4210-b760-995f1f15774f service cinder] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:12.074565 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c44eb6-f156-42ff-b70f-6c7a90562419 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:12.088671 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624f4009-f7ab-44ce-814a-c70f05c0ad79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:12.089556 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f4009-f7ab-44ce-814a-c70f05c0ad79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:12.089854 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f4009-f7ab-44ce-814a-c70f05c0ad79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:12.090208 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f4009-f7ab-44ce-814a-c70f05c0ad79 admin admin] PATH_INFO: `/v3/projects/7eb06b7c6795491899d9e0ca42564344` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:12.093060 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624f4009-f7ab-44ce-814a-c70f05c0ad79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7eb06b7c6795491899d9e0ca42564344)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:12.094968 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624f4009-f7ab-44ce-814a-c70f05c0ad79 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:12.102801 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:12.124581 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e762e14-6e93-4474-b31a-3389ef63d83c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:12.125641 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e762e14-6e93-4474-b31a-3389ef63d83c admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:12.126322 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e762e14-6e93-4474-b31a-3389ef63d83c admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:12.126680 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e762e14-6e93-4474-b31a-3389ef63d83c admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:12.128043 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e762e14-6e93-4474-b31a-3389ef63d83c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:12.131004 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e762e14-6e93-4474-b31a-3389ef63d83c admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:12.171794 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1063/2154] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:58:11 2022] POST /identity/v3/users => generated 288 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:58:12.176865 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e762e14-6e93-4474-b31a-3389ef63d83c admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:12.186222 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-624f4009-f7ab-44ce-814a-c70f05c0ad79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7eb06b7c6795491899d9e0ca42564344', 'request_id': 'req-624f4009-f7ab-44ce-814a-c70f05c0ad79'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:12.193620 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e762e14-6e93-4474-b31a-3389ef63d83c admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:12.202541 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-624f4009-f7ab-44ce-814a-c70f05c0ad79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7eb06b7c6795491899d9e0ca42564344', 'request_id': 'req-624f4009-f7ab-44ce-814a-c70f05c0ad79'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:12.217601 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-624f4009-f7ab-44ce-814a-c70f05c0ad79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7eb06b7c6795491899d9e0ca42564344', 'request_id': 'req-624f4009-f7ab-44ce-814a-c70f05c0ad79'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:12.237660 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-624f4009-f7ab-44ce-814a-c70f05c0ad79 admin admin] The token cache is being invalidate because project 7eb06b7c6795491899d9e0ca42564344 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:58:12.237953 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-624f4009-f7ab-44ce-814a-c70f05c0ad79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:12.238643 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-624f4009-f7ab-44ce-814a-c70f05c0ad79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:12.240285 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1092/2155] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:58:11 2022] DELETE /identity/v3/projects/7eb06b7c6795491899d9e0ca42564344 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:12.247601 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b0783-b119-44ba-a66b-0965312948ea None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:12.247890 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b0783-b119-44ba-a66b-0965312948ea None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:12.248177 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b0783-b119-44ba-a66b-0965312948ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:12.286014 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-6f1b0783-b119-44ba-a66b-0965312948ea None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:12.288708 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-6f1b0783-b119-44ba-a66b-0965312948ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:12.311077 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:12.340451 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff42e7f0-fc0e-4b48-a9c9-44d764214b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:12.341176 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42e7f0-fc0e-4b48-a9c9-44d764214b08 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:12.341461 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42e7f0-fc0e-4b48-a9c9-44d764214b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:12.346133 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42e7f0-fc0e-4b48-a9c9-44d764214b08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:12.347143 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff42e7f0-fc0e-4b48-a9c9-44d764214b08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:12.349673 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff42e7f0-fc0e-4b48-a9c9-44d764214b08 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:12.369029 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1064/2156] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:58:12 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:58:12.373017 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff42e7f0-fc0e-4b48-a9c9-44d764214b08 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:12.375076 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-ff42e7f0-fc0e-4b48-a9c9-44d764214b08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:12.427571 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff42e7f0-fc0e-4b48-a9c9-44d764214b08 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:12.441732 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de28caf-fd93-4451-880a-71dad2fb0ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:12.444201 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de28caf-fd93-4451-880a-71dad2fb0ccc admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:12.444201 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de28caf-fd93-4451-880a-71dad2fb0ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:12.444201 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de28caf-fd93-4451-880a-71dad2fb0ccc admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:12.446801 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de28caf-fd93-4451-880a-71dad2fb0ccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:12.453253 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de28caf-fd93-4451-880a-71dad2fb0ccc admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:12.464150 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1093/2157] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:12.471335 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8e5b9-9786-44b3-a24d-3e06f66632fd None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:12.472139 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8e5b9-9786-44b3-a24d-3e06f66632fd None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:12.472139 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8e5b9-9786-44b3-a24d-3e06f66632fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:12.473393 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1065/2158] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:58:12 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:58:12.477392 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de28caf-fd93-4451-880a-71dad2fb0ccc admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:12.511810 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de28caf-fd93-4451-880a-71dad2fb0ccc admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:12.518197 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e189139-7e26-4127-ad3c-71e5531a992f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:12.519295 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e189139-7e26-4127-ad3c-71e5531a992f admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:12.519655 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e189139-7e26-4127-ad3c-71e5531a992f admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:12.520004 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e189139-7e26-4127-ad3c-71e5531a992f admin admin] PATH_INFO: `/v3/projects/84e10d60a48d4c64af2a64c4829250ba/users/22fb486882724dcc872d906bcbbac256/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:12.523611 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-7de8e5b9-9786-44b3-a24d-3e06f66632fd None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:12.524840 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-7de8e5b9-9786-44b3-a24d-3e06f66632fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:12.538915 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e189139-7e26-4127-ad3c-71e5531a992f admin admin] RBAC: Authorizing `identity:create_grant(project_id=84e10d60a48d4c64af2a64c4829250ba, user_id=22fb486882724dcc872d906bcbbac256, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:12.540806 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1094/2159] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:12.542351 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e189139-7e26-4127-ad3c-71e5531a992f admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:12.544640 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8e5b9-9786-44b3-a24d-3e06f66632fd None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:12.557655 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8e5b9-9786-44b3-a24d-3e06f66632fd None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:12.571041 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1066/2160] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:58:12 2022] PUT /identity/v3/projects/84e10d60a48d4c64af2a64c4829250ba/users/22fb486882724dcc872d906bcbbac256/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 01:58:12.583634 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde083b5-3540-40cb-b6a3-d9ad22b2fba5 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:12.584011 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde083b5-3540-40cb-b6a3-d9ad22b2fba5 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:12.584351 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde083b5-3540-40cb-b6a3-d9ad22b2fba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:12.629445 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-cde083b5-3540-40cb-b6a3-d9ad22b2fba5 None None] MFA Rules not processed for user `22fb486882724dcc872d906bcbbac256`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:12.630676 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-cde083b5-3540-40cb-b6a3-d9ad22b2fba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:12.670495 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9cc424-2c93-4834-bbb1-e38a0ab69449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:12.671479 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9cc424-2c93-4834-bbb1-e38a0ab69449 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:12.671910 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9cc424-2c93-4834-bbb1-e38a0ab69449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:12.672353 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9cc424-2c93-4834-bbb1-e38a0ab69449 admin admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:12.673568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9cc424-2c93-4834-bbb1-e38a0ab69449 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:58:12.674019 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9cc424-2c93-4834-bbb1-e38a0ab69449 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:12.676004 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9cc424-2c93-4834-bbb1-e38a0ab69449 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:12.689112 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1095/2161] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 01:58:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:58:12.693743 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9cc424-2c93-4834-bbb1-e38a0ab69449 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:12.704462 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9cc424-2c93-4834-bbb1-e38a0ab69449 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:12.706874 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1067/2162] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:12.712397 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7bffd6-4c9d-4b1a-9a35-3aff9d845996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:12.713062 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bffd6-4c9d-4b1a-9a35-3aff9d845996 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:12.713343 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bffd6-4c9d-4b1a-9a35-3aff9d845996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:12.713618 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bffd6-4c9d-4b1a-9a35-3aff9d845996 admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:12.714317 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7bffd6-4c9d-4b1a-9a35-3aff9d845996 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:12.716407 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7bffd6-4c9d-4b1a-9a35-3aff9d845996 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:12.718892 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde083b5-3540-40cb-b6a3-d9ad22b2fba5 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:12.720974 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-cde083b5-3540-40cb-b6a3-d9ad22b2fba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:12.750746 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1096/2163] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:58:12 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:58:12.755036 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7bffd6-4c9d-4b1a-9a35-3aff9d845996 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:12.764579 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:12.811146 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde083b5-3540-40cb-b6a3-d9ad22b2fba5 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:12.839740 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7bffd6-4c9d-4b1a-9a35-3aff9d845996 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:12.844576 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f52bb-ed27-489c-96ec-6d3a3adb5334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:12.845317 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f52bb-ed27-489c-96ec-6d3a3adb5334 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:12.845660 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f52bb-ed27-489c-96ec-6d3a3adb5334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:12.846329 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f52bb-ed27-489c-96ec-6d3a3adb5334 admin admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:12.846756 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63f52bb-ed27-489c-96ec-6d3a3adb5334 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:12.849034 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63f52bb-ed27-489c-96ec-6d3a3adb5334 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:12.885351 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1097/2164] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:58:12 2022] POST /identity/v3/users => generated 292 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:58:12.889764 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f52bb-ed27-489c-96ec-6d3a3adb5334 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:12.898235 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:12.901468 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c405c-73e0-4f1d-8f8b-c690f892fc0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4e83df33b64a3ebf28f54e2d94fe63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:12.902141 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c405c-73e0-4f1d-8f8b-c690f892fc0b service nova] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:12.902424 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c405c-73e0-4f1d-8f8b-c690f892fc0b service nova] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:12.902696 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c405c-73e0-4f1d-8f8b-c690f892fc0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:12.905266 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f52bb-ed27-489c-96ec-6d3a3adb5334 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:12.912592 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc708f4f-df9c-4ad0-bcf8-d97bfe6b0086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:12.913906 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc708f4f-df9c-4ad0-bcf8-d97bfe6b0086 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:12.914626 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc708f4f-df9c-4ad0-bcf8-d97bfe6b0086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:12.915000 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc708f4f-df9c-4ad0-bcf8-d97bfe6b0086 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:12.915930 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc708f4f-df9c-4ad0-bcf8-d97bfe6b0086 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:12.917737 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc708f4f-df9c-4ad0-bcf8-d97bfe6b0086 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:12.934112 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1098/2165] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:58:12 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:58:12.935341 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc708f4f-df9c-4ad0-bcf8-d97bfe6b0086 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:12.946796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc708f4f-df9c-4ad0-bcf8-d97bfe6b0086 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:12.952586 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b9be1-cfdd-4d76-a337-5553105430ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:12.953554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b9be1-cfdd-4d76-a337-5553105430ce admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:12.953797 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b9be1-cfdd-4d76-a337-5553105430ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:12.954360 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b9be1-cfdd-4d76-a337-5553105430ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:12.955632 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339b9be1-cfdd-4d76-a337-5553105430ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:12.957580 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339b9be1-cfdd-4d76-a337-5553105430ce admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:12.973439 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5c405c-73e0-4f1d-8f8b-c690f892fc0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:12.975083 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5c405c-73e0-4f1d-8f8b-c690f892fc0b service nova] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:12.975577 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1099/2166] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:58:12 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:58:12.976679 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b9be1-cfdd-4d76-a337-5553105430ce admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:12.990277 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b9be1-cfdd-4d76-a337-5553105430ce admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:12.997082 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd5d47b-1d18-427d-a25f-b17b305bd948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:12.998334 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd5d47b-1d18-427d-a25f-b17b305bd948 admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:12.998763 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd5d47b-1d18-427d-a25f-b17b305bd948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:12.999395 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1068/2167] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:58:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:58:12.999925 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd5d47b-1d18-427d-a25f-b17b305bd948 admin admin] PATH_INFO: `/v3/projects/13f3c48049e14705992e784ca7f835c4/users/a3987b949d174acbb0b3f13d0028fa2b/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:13.020445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd5d47b-1d18-427d-a25f-b17b305bd948 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13f3c48049e14705992e784ca7f835c4, user_id=a3987b949d174acbb0b3f13d0028fa2b, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:13.024640 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd5d47b-1d18-427d-a25f-b17b305bd948 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:13.041726 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1100/2168] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:58:12 2022] PUT /identity/v3/projects/13f3c48049e14705992e784ca7f835c4/users/a3987b949d174acbb0b3f13d0028fa2b/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:13.053955 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61ddff-df5f-4a3b-b770-5bace675ec17 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:13.053955 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61ddff-df5f-4a3b-b770-5bace675ec17 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:13.054370 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61ddff-df5f-4a3b-b770-5bace675ec17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:13.104325 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-0d61ddff-df5f-4a3b-b770-5bace675ec17 None None] MFA Rules not processed for user `a3987b949d174acbb0b3f13d0028fa2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:13.105677 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-0d61ddff-df5f-4a3b-b770-5bace675ec17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:13.210363 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:13.225576 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1069/2169] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:13.237022 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd5d47b-1d18-427d-a25f-b17b305bd948 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:13.240764 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-0bd5d47b-1d18-427d-a25f-b17b305bd948 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:13.280327 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d61ddff-df5f-4a3b-b770-5bace675ec17 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:13.329518 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd5d47b-1d18-427d-a25f-b17b305bd948 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:13.334532 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ace64c6-7810-492a-a362-1a3ebdc7efcb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:13.335641 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace64c6-7810-492a-a362-1a3ebdc7efcb service cinder] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:13.336027 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace64c6-7810-492a-a362-1a3ebdc7efcb service cinder] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:13.336405 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace64c6-7810-492a-a362-1a3ebdc7efcb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:13.346031 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ace64c6-7810-492a-a362-1a3ebdc7efcb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:13.348019 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ace64c6-7810-492a-a362-1a3ebdc7efcb service cinder] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:13.368963 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1101/2170] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:58:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:58:13.383729 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d61ddff-df5f-4a3b-b770-5bace675ec17 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:13.389046 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca62277-824c-49c3-825e-11789dc42e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:13.390098 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca62277-824c-49c3-825e-11789dc42e69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:13.390388 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca62277-824c-49c3-825e-11789dc42e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:13.390724 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca62277-824c-49c3-825e-11789dc42e69 admin admin] PATH_INFO: `/v3/users/22fb486882724dcc872d906bcbbac256` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:13.392163 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca62277-824c-49c3-825e-11789dc42e69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22fb486882724dcc872d906bcbbac256)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:13.395539 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca62277-824c-49c3-825e-11789dc42e69 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:13.465340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-1ca62277-824c-49c3-825e-11789dc42e69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22fb486882724dcc872d906bcbbac256'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:13.484044 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-1ca62277-824c-49c3-825e-11789dc42e69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22fb486882724dcc872d906bcbbac256'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:13.497400 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-1ca62277-824c-49c3-825e-11789dc42e69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22fb486882724dcc872d906bcbbac256'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:13.512355 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-1ca62277-824c-49c3-825e-11789dc42e69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22fb486882724dcc872d906bcbbac256'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:13.515439 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-1ca62277-824c-49c3-825e-11789dc42e69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22fb486882724dcc872d906bcbbac256'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:13.530057 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1070/2171] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:58:13 2022] DELETE /identity/v3/users/22fb486882724dcc872d906bcbbac256 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:13.534617 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2f9a4-b080-4078-876b-5d717d1605ec None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:13.535617 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2f9a4-b080-4078-876b-5d717d1605ec None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:13.535901 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2f9a4-b080-4078-876b-5d717d1605ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:13.582790 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-24c2f9a4-b080-4078-876b-5d717d1605ec None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:13.584699 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-24c2f9a4-b080-4078-876b-5d717d1605ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:13.779684 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1102/2172] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 01:58:13.788610 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca62277-824c-49c3-825e-11789dc42e69 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:13.790357 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-1ca62277-824c-49c3-825e-11789dc42e69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:13.945407 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca62277-824c-49c3-825e-11789dc42e69 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:13.963670 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d8fa7-c29a-44f2-b822-7f5e1db34434 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:13.964397 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d8fa7-c29a-44f2-b822-7f5e1db34434 service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:13.968315 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d8fa7-c29a-44f2-b822-7f5e1db34434 service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:13.968685 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d8fa7-c29a-44f2-b822-7f5e1db34434 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:14.014922 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93d8fa7-c29a-44f2-b822-7f5e1db34434 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:14.016326 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93d8fa7-c29a-44f2-b822-7f5e1db34434 service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:14.059412 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1071/2173] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:58:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:58:14.519204 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c2f9a4-b080-4078-876b-5d717d1605ec None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:14.520949 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-24c2f9a4-b080-4078-876b-5d717d1605ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:14.542212 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c2f9a4-b080-4078-876b-5d717d1605ec None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:14.548185 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc96c4-5f12-465d-a9ef-4c1505e6ded5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:14.549285 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc96c4-5f12-465d-a9ef-4c1505e6ded5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:14.549629 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc96c4-5f12-465d-a9ef-4c1505e6ded5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:14.550013 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc96c4-5f12-465d-a9ef-4c1505e6ded5 admin admin] PATH_INFO: `/v3/projects/84e10d60a48d4c64af2a64c4829250ba` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:14.551327 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc96c4-5f12-465d-a9ef-4c1505e6ded5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84e10d60a48d4c64af2a64c4829250ba)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:14.554015 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc96c4-5f12-465d-a9ef-4c1505e6ded5 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:14.620236 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-cddc96c4-5f12-465d-a9ef-4c1505e6ded5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84e10d60a48d4c64af2a64c4829250ba', 'request_id': 'req-cddc96c4-5f12-465d-a9ef-4c1505e6ded5'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:14.627890 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-cddc96c4-5f12-465d-a9ef-4c1505e6ded5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84e10d60a48d4c64af2a64c4829250ba', 'request_id': 'req-cddc96c4-5f12-465d-a9ef-4c1505e6ded5'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:14.633794 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:14.640657 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-cddc96c4-5f12-465d-a9ef-4c1505e6ded5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84e10d60a48d4c64af2a64c4829250ba', 'request_id': 'req-cddc96c4-5f12-465d-a9ef-4c1505e6ded5'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:14.654205 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-cddc96c4-5f12-465d-a9ef-4c1505e6ded5 admin admin] The token cache is being invalidate because project 84e10d60a48d4c64af2a64c4829250ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:58:14.654906 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-cddc96c4-5f12-465d-a9ef-4c1505e6ded5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:14.655878 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-cddc96c4-5f12-465d-a9ef-4c1505e6ded5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:14.657920 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1103/2174] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:58:14 2022] DELETE /identity/v3/projects/84e10d60a48d4c64af2a64c4829250ba => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:14.662647 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd481469-9626-4ccf-97ca-353a0def0a10 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:14.663122 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd481469-9626-4ccf-97ca-353a0def0a10 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:14.663539 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd481469-9626-4ccf-97ca-353a0def0a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:14.686708 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-fd481469-9626-4ccf-97ca-353a0def0a10 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:14.688128 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-fd481469-9626-4ccf-97ca-353a0def0a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:14.708957 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:14.767364 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1072/2175] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:14.773132 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930bc737-6186-48e0-a768-f8f2f935f33f None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:14.774130 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930bc737-6186-48e0-a768-f8f2f935f33f None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:14.774130 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930bc737-6186-48e0-a768-f8f2f935f33f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:14.794868 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-930bc737-6186-48e0-a768-f8f2f935f33f None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:14.796710 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-930bc737-6186-48e0-a768-f8f2f935f33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:14.807829 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1104/2176] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 01:58:14.812111 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd481469-9626-4ccf-97ca-353a0def0a10 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:14.823322 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd481469-9626-4ccf-97ca-353a0def0a10 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:14.828245 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c118b4c8-f304-47fb-b9f5-c8bf3422e916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:14.829046 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c118b4c8-f304-47fb-b9f5-c8bf3422e916 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:14.829428 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c118b4c8-f304-47fb-b9f5-c8bf3422e916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:14.829807 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c118b4c8-f304-47fb-b9f5-c8bf3422e916 admin admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:14.830649 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c118b4c8-f304-47fb-b9f5-c8bf3422e916 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:58:14.831056 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c118b4c8-f304-47fb-b9f5-c8bf3422e916 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:14.832926 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c118b4c8-f304-47fb-b9f5-c8bf3422e916 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:14.852572 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1073/2177] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 01:58:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:58:14.856855 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930bc737-6186-48e0-a768-f8f2f935f33f None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:14.866781 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930bc737-6186-48e0-a768-f8f2f935f33f None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:14.872066 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0443aec-3b1a-4717-94d1-8813d088771c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:14.872618 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0443aec-3b1a-4717-94d1-8813d088771c admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:14.872898 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0443aec-3b1a-4717-94d1-8813d088771c admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:14.873718 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0443aec-3b1a-4717-94d1-8813d088771c admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:14.873986 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0443aec-3b1a-4717-94d1-8813d088771c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:14.876207 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0443aec-3b1a-4717-94d1-8813d088771c admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:14.913733 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1105/2178] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:58:14 2022] POST /identity/v3/projects => generated 365 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:58:14.915241 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c118b4c8-f304-47fb-b9f5-c8bf3422e916 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:14.930179 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c118b4c8-f304-47fb-b9f5-c8bf3422e916 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:15.005517 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9eb419-1af8-42ac-bd9a-8546dbc5ae75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:15.006261 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9eb419-1af8-42ac-bd9a-8546dbc5ae75 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:15.006550 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9eb419-1af8-42ac-bd9a-8546dbc5ae75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:15.006831 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9eb419-1af8-42ac-bd9a-8546dbc5ae75 admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:15.007599 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9eb419-1af8-42ac-bd9a-8546dbc5ae75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:15.009640 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9eb419-1af8-42ac-bd9a-8546dbc5ae75 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:15.039723 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1074/2179] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:58:14 2022] POST /identity/v3/users => generated 283 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:58:15.043767 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0443aec-3b1a-4717-94d1-8813d088771c admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:15.058127 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0443aec-3b1a-4717-94d1-8813d088771c admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:15.062830 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef674dff-ae09-41c2-9228-93a9bfaeaed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:15.063683 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef674dff-ae09-41c2-9228-93a9bfaeaed6 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:15.063973 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef674dff-ae09-41c2-9228-93a9bfaeaed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:15.064238 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef674dff-ae09-41c2-9228-93a9bfaeaed6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:15.064982 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef674dff-ae09-41c2-9228-93a9bfaeaed6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:15.067561 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef674dff-ae09-41c2-9228-93a9bfaeaed6 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:15.084441 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1106/2180] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:58:15 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:58:15.088313 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9eb419-1af8-42ac-bd9a-8546dbc5ae75 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:15.100585 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9eb419-1af8-42ac-bd9a-8546dbc5ae75 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:15.108888 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf89780-fd1b-4b85-91b9-1b7acc8b961c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:15.109590 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf89780-fd1b-4b85-91b9-1b7acc8b961c admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:15.109878 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf89780-fd1b-4b85-91b9-1b7acc8b961c admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:15.110183 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf89780-fd1b-4b85-91b9-1b7acc8b961c admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:15.112214 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf89780-fd1b-4b85-91b9-1b7acc8b961c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:15.115067 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf89780-fd1b-4b85-91b9-1b7acc8b961c admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:15.137099 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1075/2181] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:58:15 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:58:15.141304 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef674dff-ae09-41c2-9228-93a9bfaeaed6 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:15.154099 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef674dff-ae09-41c2-9228-93a9bfaeaed6 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:15.165322 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e8581-71d2-47cd-8904-7a230f336c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:15.166293 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e8581-71d2-47cd-8904-7a230f336c06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:15.166636 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e8581-71d2-47cd-8904-7a230f336c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:15.166975 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e8581-71d2-47cd-8904-7a230f336c06 admin admin] PATH_INFO: `/v3/projects/afaae27310d54c62ba9ac5b02db97b19/users/7e05437bac434213a4c2f7a7e437c221/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:15.193207 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101e8581-71d2-47cd-8904-7a230f336c06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afaae27310d54c62ba9ac5b02db97b19, user_id=7e05437bac434213a4c2f7a7e437c221, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:15.196198 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101e8581-71d2-47cd-8904-7a230f336c06 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:15.224572 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1107/2182] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:58:15 2022] PUT /identity/v3/projects/afaae27310d54c62ba9ac5b02db97b19/users/7e05437bac434213a4c2f7a7e437c221/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:15.234351 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995660ca-30a5-4785-bd6e-d7a2797892f4 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:15.234647 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995660ca-30a5-4785-bd6e-d7a2797892f4 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:15.234929 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995660ca-30a5-4785-bd6e-d7a2797892f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:15.287142 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-995660ca-30a5-4785-bd6e-d7a2797892f4 None None] MFA Rules not processed for user `7e05437bac434213a4c2f7a7e437c221`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:15.296384 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-995660ca-30a5-4785-bd6e-d7a2797892f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:15.481969 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1076/2183] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:15 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:15.483636 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e8581-71d2-47cd-8904-7a230f336c06 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:15.609836 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e8581-71d2-47cd-8904-7a230f336c06 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:15.615541 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654f8a0-5aee-481f-a830-83451e58fbe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:15.616586 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654f8a0-5aee-481f-a830-83451e58fbe9 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:15.617148 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654f8a0-5aee-481f-a830-83451e58fbe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:15.617886 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654f8a0-5aee-481f-a830-83451e58fbe9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:15.619101 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e654f8a0-5aee-481f-a830-83451e58fbe9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:15.621522 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e654f8a0-5aee-481f-a830-83451e58fbe9 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:15.682045 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995660ca-30a5-4785-bd6e-d7a2797892f4 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:15.694801 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1108/2184] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:58:15 2022] POST /identity/v3/projects => generated 365 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:58:15.703692 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995660ca-30a5-4785-bd6e-d7a2797892f4 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:15.751193 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:15.860788 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86c67b-25bf-46ce-bf05-6f4e453bc50b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:15.861749 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86c67b-25bf-46ce-bf05-6f4e453bc50b admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:15.862183 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86c67b-25bf-46ce-bf05-6f4e453bc50b admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:15.864185 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86c67b-25bf-46ce-bf05-6f4e453bc50b admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:15.865078 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86c67b-25bf-46ce-bf05-6f4e453bc50b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:15.867688 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86c67b-25bf-46ce-bf05-6f4e453bc50b admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:15.900732 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1077/2185] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:58:15 2022] POST /identity/v3/users => generated 289 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:58:15.904877 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654f8a0-5aee-481f-a830-83451e58fbe9 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:15.919248 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654f8a0-5aee-481f-a830-83451e58fbe9 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:15.927475 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd1753-b8f5-4a70-8a72-59efeb57b2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:15.929794 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd1753-b8f5-4a70-8a72-59efeb57b2b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:15.930644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd1753-b8f5-4a70-8a72-59efeb57b2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:15.935395 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd1753-b8f5-4a70-8a72-59efeb57b2b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:15.936238 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafd1753-b8f5-4a70-8a72-59efeb57b2b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:15.938208 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafd1753-b8f5-4a70-8a72-59efeb57b2b4 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:15.952818 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1109/2186] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:58:15 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:58:15.957236 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86c67b-25bf-46ce-bf05-6f4e453bc50b admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:15.968457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86c67b-25bf-46ce-bf05-6f4e453bc50b admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:15.974876 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26fefb2-eb11-475c-80a5-fe03090a6304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:15.975899 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26fefb2-eb11-475c-80a5-fe03090a6304 admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:15.976397 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26fefb2-eb11-475c-80a5-fe03090a6304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:15.976765 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26fefb2-eb11-475c-80a5-fe03090a6304 admin admin] PATH_INFO: `/v3/projects/a0a59bd5430d416f8976aa7aaf41834c/users/1ac106e9c3a944ee8a947557ea129143/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:16.005788 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26fefb2-eb11-475c-80a5-fe03090a6304 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0a59bd5430d416f8976aa7aaf41834c, user_id=1ac106e9c3a944ee8a947557ea129143, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:16.009017 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26fefb2-eb11-475c-80a5-fe03090a6304 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:16.033063 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd1753-b8f5-4a70-8a72-59efeb57b2b4 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:16.035079 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1078/2187] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:58:15 2022] PUT /identity/v3/projects/a0a59bd5430d416f8976aa7aaf41834c/users/1ac106e9c3a944ee8a947557ea129143/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:16.055749 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd1753-b8f5-4a70-8a72-59efeb57b2b4 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:16.057583 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867519f2-76dc-4fc1-a80b-6915acd3713e None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:16.057885 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867519f2-76dc-4fc1-a80b-6915acd3713e None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:16.058427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867519f2-76dc-4fc1-a80b-6915acd3713e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:16.090311 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-867519f2-76dc-4fc1-a80b-6915acd3713e None None] MFA Rules not processed for user `3d72a03c4eec4412856ee48cb967c46d`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:16.091608 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-867519f2-76dc-4fc1-a80b-6915acd3713e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:16.137320 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:16.146306 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d84e6-4966-490a-8862-945615cbcef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:16.147346 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d84e6-4966-490a-8862-945615cbcef6 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:16.147869 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d84e6-4966-490a-8862-945615cbcef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:16.148500 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d84e6-4966-490a-8862-945615cbcef6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:16.149548 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529d84e6-4966-490a-8862-945615cbcef6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:16.151641 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529d84e6-4966-490a-8862-945615cbcef6 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:16.167139 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1110/2188] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:58:16.168396 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d84e6-4966-490a-8862-945615cbcef6 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:16.178176 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1079/2189] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:16.179957 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d84e6-4966-490a-8862-945615cbcef6 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:16.183496 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867519f2-76dc-4fc1-a80b-6915acd3713e None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:16.185432 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-867519f2-76dc-4fc1-a80b-6915acd3713e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:16.186035 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d43691c-87ad-46c4-bfa9-0f4ec37032c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:16.186933 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d43691c-87ad-46c4-bfa9-0f4ec37032c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:16.187249 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d43691c-87ad-46c4-bfa9-0f4ec37032c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:16.187513 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d43691c-87ad-46c4-bfa9-0f4ec37032c2 admin admin] PATH_INFO: `/v3/projects/a0a59bd5430d416f8976aa7aaf41834c/users/1ac106e9c3a944ee8a947557ea129143/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:16.190189 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d43691c-87ad-46c4-bfa9-0f4ec37032c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0a59bd5430d416f8976aa7aaf41834c, user_id=1ac106e9c3a944ee8a947557ea129143, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:16.193114 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d43691c-87ad-46c4-bfa9-0f4ec37032c2 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:16.205315 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1111/2190] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:58:16 2022] PUT /identity/v3/projects/a0a59bd5430d416f8976aa7aaf41834c/users/1ac106e9c3a944ee8a947557ea129143/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:16.209251 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867519f2-76dc-4fc1-a80b-6915acd3713e None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:16.219309 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0d8d0b-2d90-4685-a051-cfcfc5473fdd None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:16.219841 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0d8d0b-2d90-4685-a051-cfcfc5473fdd None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:16.220271 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0d8d0b-2d90-4685-a051-cfcfc5473fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:16.277642 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-1c0d8d0b-2d90-4685-a051-cfcfc5473fdd None None] MFA Rules not processed for user `1ac106e9c3a944ee8a947557ea129143`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:16.279094 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-1c0d8d0b-2d90-4685-a051-cfcfc5473fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:16.333339 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb5583-94a5-40ea-8da9-8e70a7b7a515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:16.334163 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb5583-94a5-40ea-8da9-8e70a7b7a515 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:16.334508 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb5583-94a5-40ea-8da9-8e70a7b7a515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:16.334856 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb5583-94a5-40ea-8da9-8e70a7b7a515 admin admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:16.336159 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eb5583-94a5-40ea-8da9-8e70a7b7a515 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:16.337701 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eb5583-94a5-40ea-8da9-8e70a7b7a515 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:16.353883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:16.371945 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1080/2191] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:58:16 2022] POST /identity/v3/projects => generated 383 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:58:16.375867 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb5583-94a5-40ea-8da9-8e70a7b7a515 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:16.388539 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1112/2192] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:16 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 01:58:16.401017 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0d8d0b-2d90-4685-a051-cfcfc5473fdd None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:16.402524 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-1c0d8d0b-2d90-4685-a051-cfcfc5473fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:16.469898 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb5583-94a5-40ea-8da9-8e70a7b7a515 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:16.490880 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0d8d0b-2d90-4685-a051-cfcfc5473fdd None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:16.501815 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6fac57-0f9e-434b-81fe-3cd800402add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:16.502915 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6fac57-0f9e-434b-81fe-3cd800402add admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:16.503290 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6fac57-0f9e-434b-81fe-3cd800402add admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:16.503570 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6fac57-0f9e-434b-81fe-3cd800402add admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:16.504338 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6fac57-0f9e-434b-81fe-3cd800402add admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:16.506447 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6fac57-0f9e-434b-81fe-3cd800402add admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:16.534873 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1081/2193] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:58:16 2022] POST /identity/v3/users => generated 298 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:58:16.538747 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6fac57-0f9e-434b-81fe-3cd800402add admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:16.540524 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-0a6fac57-0f9e-434b-81fe-3cd800402add admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:16.555575 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7037092c-cb42-4fa1-8c2d-967be7f0867c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:16.556541 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7037092c-cb42-4fa1-8c2d-967be7f0867c service cinder] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:16.557031 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7037092c-cb42-4fa1-8c2d-967be7f0867c service cinder] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:16.557512 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7037092c-cb42-4fa1-8c2d-967be7f0867c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:16.561131 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6fac57-0f9e-434b-81fe-3cd800402add admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:16.567806 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88105f56-aaa0-43b5-98bf-b6f180d3bd8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:16.568698 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88105f56-aaa0-43b5-98bf-b6f180d3bd8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:16.568981 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88105f56-aaa0-43b5-98bf-b6f180d3bd8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:16.569255 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88105f56-aaa0-43b5-98bf-b6f180d3bd8b admin admin] PATH_INFO: `/v3/users/a3987b949d174acbb0b3f13d0028fa2b` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:16.570618 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88105f56-aaa0-43b5-98bf-b6f180d3bd8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3987b949d174acbb0b3f13d0028fa2b)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:16.573629 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88105f56-aaa0-43b5-98bf-b6f180d3bd8b admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:16.595403 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:16.639247 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7037092c-cb42-4fa1-8c2d-967be7f0867c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:16.640698 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-88105f56-aaa0-43b5-98bf-b6f180d3bd8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3987b949d174acbb0b3f13d0028fa2b'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:16.640826 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7037092c-cb42-4fa1-8c2d-967be7f0867c service cinder] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:16.655275 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1113/2194] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:58:16 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 01:58:16.656139 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7037092c-cb42-4fa1-8c2d-967be7f0867c service cinder] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:16.666113 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-88105f56-aaa0-43b5-98bf-b6f180d3bd8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3987b949d174acbb0b3f13d0028fa2b'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:16.668974 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7037092c-cb42-4fa1-8c2d-967be7f0867c service cinder] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:16.676059 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e241e-1d85-4180-a74b-d6aa490c31fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:16.677385 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e241e-1d85-4180-a74b-d6aa490c31fd admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:16.677737 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e241e-1d85-4180-a74b-d6aa490c31fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:16.678333 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e241e-1d85-4180-a74b-d6aa490c31fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:16.679237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8e241e-1d85-4180-a74b-d6aa490c31fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:16.681181 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8e241e-1d85-4180-a74b-d6aa490c31fd admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:16.684192 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-88105f56-aaa0-43b5-98bf-b6f180d3bd8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3987b949d174acbb0b3f13d0028fa2b'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:16.694912 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1114/2195] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:58:16.698632 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-88105f56-aaa0-43b5-98bf-b6f180d3bd8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3987b949d174acbb0b3f13d0028fa2b'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:16.699574 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-88105f56-aaa0-43b5-98bf-b6f180d3bd8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3987b949d174acbb0b3f13d0028fa2b'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:16.699868 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e241e-1d85-4180-a74b-d6aa490c31fd admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:16.703312 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-cc8e241e-1d85-4180-a74b-d6aa490c31fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:16.707610 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1082/2196] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:58:16 2022] DELETE /identity/v3/users/a3987b949d174acbb0b3f13d0028fa2b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:16.712429 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8886eb-fff0-4ea3-a4dc-cc116fa84f76 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:16.713013 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8886eb-fff0-4ea3-a4dc-cc116fa84f76 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:16.713433 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8886eb-fff0-4ea3-a4dc-cc116fa84f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:16.724787 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e241e-1d85-4180-a74b-d6aa490c31fd admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:16.741387 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-8d8886eb-fff0-4ea3-a4dc-cc116fa84f76 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:16.742844 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-8d8886eb-fff0-4ea3-a4dc-cc116fa84f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:16.835366 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d9fe5c-3e79-4810-9ddc-8860548914fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:16.836908 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9fe5c-3e79-4810-9ddc-8860548914fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:16.837406 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9fe5c-3e79-4810-9ddc-8860548914fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:16.837690 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9fe5c-3e79-4810-9ddc-8860548914fa admin admin] PATH_INFO: `/v3/projects/5d85388598234c5e88414a1dbf17a5f8/users/f92ee0300da141d3b8a1aec100e8c0d6/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:16.855082 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:16.858584 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d9fe5c-3e79-4810-9ddc-8860548914fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d85388598234c5e88414a1dbf17a5f8, user_id=f92ee0300da141d3b8a1aec100e8c0d6, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:16.861714 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d9fe5c-3e79-4810-9ddc-8860548914fa admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:16.866819 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1083/2197] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:16.867780 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8886eb-fff0-4ea3-a4dc-cc116fa84f76 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:16.869375 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-8d8886eb-fff0-4ea3-a4dc-cc116fa84f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:16.895310 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1115/2198] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:58:16 2022] PUT /identity/v3/projects/5d85388598234c5e88414a1dbf17a5f8/users/f92ee0300da141d3b8a1aec100e8c0d6/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 01:58:16.895310 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d9fe5c-3e79-4810-9ddc-8860548914fa admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:16.897233 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-29d9fe5c-3e79-4810-9ddc-8860548914fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:16.955548 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8886eb-fff0-4ea3-a4dc-cc116fa84f76 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:16.985100 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d9fe5c-3e79-4810-9ddc-8860548914fa admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:16.989130 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2407776-6595-424b-b7dd-3013fddc3f19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:16.989807 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2407776-6595-424b-b7dd-3013fddc3f19 service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:16.990088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2407776-6595-424b-b7dd-3013fddc3f19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:16.990359 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2407776-6595-424b-b7dd-3013fddc3f19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:17.013706 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd0ec5d-d16a-4300-a10e-d9be47c60984 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:17.014434 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd0ec5d-d16a-4300-a10e-d9be47c60984 service cinder] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:17.015494 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd0ec5d-d16a-4300-a10e-d9be47c60984 service cinder] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:17.015803 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd0ec5d-d16a-4300-a10e-d9be47c60984 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:17.017780 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-bbd0ec5d-d16a-4300-a10e-d9be47c60984 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:17.050860 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:17.064404 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2407776-6595-424b-b7dd-3013fddc3f19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:17.066292 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2407776-6595-424b-b7dd-3013fddc3f19 service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:17.085204 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd0ec5d-d16a-4300-a10e-d9be47c60984 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:17.086750 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd0ec5d-d16a-4300-a10e-d9be47c60984 service cinder] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:17.090934 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1116/2199] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:58:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:58:17.092354 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2407776-6595-424b-b7dd-3013fddc3f19 service neutron] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:17.097246 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1084/2200] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:58:16 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:58:17.098149 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd0ec5d-d16a-4300-a10e-d9be47c60984 service cinder] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:17.100396 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-bbd0ec5d-d16a-4300-a10e-d9be47c60984 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:17.113794 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2407776-6595-424b-b7dd-3013fddc3f19 service neutron] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:17.114858 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd0ec5d-d16a-4300-a10e-d9be47c60984 service cinder] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:17.120841 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3945fa93-9163-4230-a7ad-0571437246f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:17.121683 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945fa93-9163-4230-a7ad-0571437246f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:17.122098 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945fa93-9163-4230-a7ad-0571437246f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:17.122825 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945fa93-9163-4230-a7ad-0571437246f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:17.125039 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-3945fa93-9163-4230-a7ad-0571437246f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:17.127816 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdec2c9-b587-4bcc-aa50-44bdcd2499af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:17.129450 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdec2c9-b587-4bcc-aa50-44bdcd2499af admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:17.129639 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdec2c9-b587-4bcc-aa50-44bdcd2499af admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:17.129923 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdec2c9-b587-4bcc-aa50-44bdcd2499af admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:17.131109 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdec2c9-b587-4bcc-aa50-44bdcd2499af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:17.134594 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdec2c9-b587-4bcc-aa50-44bdcd2499af admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:17.151915 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1117/2201] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:58:17 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:58:17.156203 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdec2c9-b587-4bcc-aa50-44bdcd2499af admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:17.168400 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdec2c9-b587-4bcc-aa50-44bdcd2499af admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:17.174305 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3572d4-3997-4e6c-8efe-326ca87b28ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:17.175492 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3572d4-3997-4e6c-8efe-326ca87b28ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:17.176031 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3572d4-3997-4e6c-8efe-326ca87b28ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:17.176315 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3572d4-3997-4e6c-8efe-326ca87b28ef admin admin] PATH_INFO: `/v3/projects/5d85388598234c5e88414a1dbf17a5f8/users/f92ee0300da141d3b8a1aec100e8c0d6/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:17.179522 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3572d4-3997-4e6c-8efe-326ca87b28ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d85388598234c5e88414a1dbf17a5f8, user_id=f92ee0300da141d3b8a1aec100e8c0d6, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:17.182955 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3572d4-3997-4e6c-8efe-326ca87b28ef admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:17.202210 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1118/2202] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:58:17 2022] PUT /identity/v3/projects/5d85388598234c5e88414a1dbf17a5f8/users/f92ee0300da141d3b8a1aec100e8c0d6/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 01:58:17.229282 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3945fa93-9163-4230-a7ad-0571437246f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:17.231545 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3945fa93-9163-4230-a7ad-0571437246f1 service cinder] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:17.243793 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1085/2203] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:58:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:58:17.711049 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3572d4-3997-4e6c-8efe-326ca87b28ef admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:17.713177 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-ba3572d4-3997-4e6c-8efe-326ca87b28ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:17.840981 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3572d4-3997-4e6c-8efe-326ca87b28ef admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:17.847481 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d958b58-0602-4ec3-b9b8-d7631c94d85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:17.848364 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d958b58-0602-4ec3-b9b8-d7631c94d85a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:17.848643 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d958b58-0602-4ec3-b9b8-d7631c94d85a admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:17.848921 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d958b58-0602-4ec3-b9b8-d7631c94d85a admin admin] PATH_INFO: `/v3/projects/13f3c48049e14705992e784ca7f835c4` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:17.850166 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d958b58-0602-4ec3-b9b8-d7631c94d85a admin admin] RBAC: Authorizing `identity:delete_project(project_id=13f3c48049e14705992e784ca7f835c4)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:17.852193 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d958b58-0602-4ec3-b9b8-d7631c94d85a admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:17.911192 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-3d958b58-0602-4ec3-b9b8-d7631c94d85a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13f3c48049e14705992e784ca7f835c4', 'request_id': 'req-3d958b58-0602-4ec3-b9b8-d7631c94d85a'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:17.918293 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-3d958b58-0602-4ec3-b9b8-d7631c94d85a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13f3c48049e14705992e784ca7f835c4', 'request_id': 'req-3d958b58-0602-4ec3-b9b8-d7631c94d85a'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:17.927460 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-3d958b58-0602-4ec3-b9b8-d7631c94d85a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13f3c48049e14705992e784ca7f835c4', 'request_id': 'req-3d958b58-0602-4ec3-b9b8-d7631c94d85a'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:17.938811 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-3d958b58-0602-4ec3-b9b8-d7631c94d85a admin admin] The token cache is being invalidate because project 13f3c48049e14705992e784ca7f835c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:58:17.939221 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-3d958b58-0602-4ec3-b9b8-d7631c94d85a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:17.940025 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-3d958b58-0602-4ec3-b9b8-d7631c94d85a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:17.942155 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1119/2204] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:58:17 2022] DELETE /identity/v3/projects/13f3c48049e14705992e784ca7f835c4 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:17.947792 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f3ef42-a05f-46b8-b400-01c4a3ae0c26 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:17.948691 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f3ef42-a05f-46b8-b400-01c4a3ae0c26 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:17.949274 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f3ef42-a05f-46b8-b400-01c4a3ae0c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:17.962600 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:17.970640 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-a1f3ef42-a05f-46b8-b400-01c4a3ae0c26 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:17.972161 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-a1f3ef42-a05f-46b8-b400-01c4a3ae0c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:18.044877 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1086/2205] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:18.057501 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0854d38a-eb01-4dbc-845a-f2a7d7c7e671 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:18.058348 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0854d38a-eb01-4dbc-845a-f2a7d7c7e671 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:18.058776 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0854d38a-eb01-4dbc-845a-f2a7d7c7e671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:18.082030 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-0854d38a-eb01-4dbc-845a-f2a7d7c7e671 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:18.083427 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-0854d38a-eb01-4dbc-845a-f2a7d7c7e671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:18.099741 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1120/2206] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:18.101237 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f3ef42-a05f-46b8-b400-01c4a3ae0c26 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:18.136781 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f3ef42-a05f-46b8-b400-01c4a3ae0c26 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:18.160605 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf50ad-7746-46ac-aa60-87195123e687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:18.161296 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf50ad-7746-46ac-aa60-87195123e687 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:18.161585 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf50ad-7746-46ac-aa60-87195123e687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:18.161855 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf50ad-7746-46ac-aa60-87195123e687 admin admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:18.163049 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf50ad-7746-46ac-aa60-87195123e687 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:58:18.163430 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf50ad-7746-46ac-aa60-87195123e687 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:18.165653 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf50ad-7746-46ac-aa60-87195123e687 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:18.188241 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1087/2207] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 01:58:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:58:18.196609 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0854d38a-eb01-4dbc-845a-f2a7d7c7e671 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:18.214464 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0854d38a-eb01-4dbc-845a-f2a7d7c7e671 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:18.219544 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05ecc5d-3b95-4f1e-9a9b-80c83244b430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:18.220649 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ecc5d-3b95-4f1e-9a9b-80c83244b430 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:18.220937 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ecc5d-3b95-4f1e-9a9b-80c83244b430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:18.221212 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ecc5d-3b95-4f1e-9a9b-80c83244b430 admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:18.222177 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05ecc5d-3b95-4f1e-9a9b-80c83244b430 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:18.224582 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05ecc5d-3b95-4f1e-9a9b-80c83244b430 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:18.274588 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1121/2208] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:58:18 2022] POST /identity/v3/projects => generated 365 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:58:18.279140 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf50ad-7746-46ac-aa60-87195123e687 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:18.294520 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf50ad-7746-46ac-aa60-87195123e687 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:18.429063 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa7b5a-3f68-4441-a855-4dd749806e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:18.431262 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa7b5a-3f68-4441-a855-4dd749806e76 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:18.431558 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa7b5a-3f68-4441-a855-4dd749806e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:18.433676 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa7b5a-3f68-4441-a855-4dd749806e76 admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:18.434436 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa7b5a-3f68-4441-a855-4dd749806e76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:18.436511 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa7b5a-3f68-4441-a855-4dd749806e76 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:18.466867 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1088/2209] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:58:18 2022] POST /identity/v3/users => generated 283 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:58:18.467940 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05ecc5d-3b95-4f1e-9a9b-80c83244b430 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:18.485135 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05ecc5d-3b95-4f1e-9a9b-80c83244b430 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:18.498228 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b703a-1812-4638-99b1-c3470a539e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:18.499059 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b703a-1812-4638-99b1-c3470a539e60 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:18.499485 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b703a-1812-4638-99b1-c3470a539e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:18.499929 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b703a-1812-4638-99b1-c3470a539e60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:18.502375 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b703a-1812-4638-99b1-c3470a539e60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:18.508995 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b703a-1812-4638-99b1-c3470a539e60 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:18.538401 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1122/2210] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:58:18 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 01:58:18.545034 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa7b5a-3f68-4441-a855-4dd749806e76 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:18.558618 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa7b5a-3f68-4441-a855-4dd749806e76 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:18.574329 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b8dd7-aeb8-4b6c-b8e2-66f84c0171c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:18.575035 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b8dd7-aeb8-4b6c-b8e2-66f84c0171c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:18.575347 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b8dd7-aeb8-4b6c-b8e2-66f84c0171c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:18.575625 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b8dd7-aeb8-4b6c-b8e2-66f84c0171c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:18.576372 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499b8dd7-aeb8-4b6c-b8e2-66f84c0171c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:18.578685 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499b8dd7-aeb8-4b6c-b8e2-66f84c0171c1 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:18.603693 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1089/2211] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:58:18 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:58:18.607358 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b703a-1812-4638-99b1-c3470a539e60 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:18.621824 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b703a-1812-4638-99b1-c3470a539e60 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:18.635493 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd7bafc-205a-4d61-8ba9-03b8026d4717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:18.636403 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd7bafc-205a-4d61-8ba9-03b8026d4717 admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:18.636685 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd7bafc-205a-4d61-8ba9-03b8026d4717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:18.637148 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd7bafc-205a-4d61-8ba9-03b8026d4717 admin admin] PATH_INFO: `/v3/projects/243b8841465e4e798163ba2f0bb1d33c/users/f3b443fd07d94499b6c9b8fa990c2d74/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:18.675476 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd7bafc-205a-4d61-8ba9-03b8026d4717 admin admin] RBAC: Authorizing `identity:create_grant(project_id=243b8841465e4e798163ba2f0bb1d33c, user_id=f3b443fd07d94499b6c9b8fa990c2d74, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:18.678619 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd7bafc-205a-4d61-8ba9-03b8026d4717 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:18.684819 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:18.705874 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1123/2212] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:58:18 2022] PUT /identity/v3/projects/243b8841465e4e798163ba2f0bb1d33c/users/f3b443fd07d94499b6c9b8fa990c2d74/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:18.716520 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87debcf9-041b-47ec-a371-a8d5c4a723e2 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:18.717297 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87debcf9-041b-47ec-a371-a8d5c4a723e2 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:18.717734 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87debcf9-041b-47ec-a371-a8d5c4a723e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:18.774040 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-87debcf9-041b-47ec-a371-a8d5c4a723e2 None None] MFA Rules not processed for user `f3b443fd07d94499b6c9b8fa990c2d74`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:18.775651 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-87debcf9-041b-47ec-a371-a8d5c4a723e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:18.936096 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1090/2213] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:18 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:18.951087 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd7bafc-205a-4d61-8ba9-03b8026d4717 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:18.955601 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-dfd7bafc-205a-4d61-8ba9-03b8026d4717 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:19.082705 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd7bafc-205a-4d61-8ba9-03b8026d4717 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:19.093584 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e2479-616d-45c5-a9be-782599c988d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:19.094416 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e2479-616d-45c5-a9be-782599c988d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:19.095840 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e2479-616d-45c5-a9be-782599c988d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:19.096211 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e2479-616d-45c5-a9be-782599c988d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:19.108590 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896e2479-616d-45c5-a9be-782599c988d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:19.110226 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896e2479-616d-45c5-a9be-782599c988d4 service cinder] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:19.132245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1124/2214] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:58:18 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:58:19.953025 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87debcf9-041b-47ec-a371-a8d5c4a723e2 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:19.955594 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-87debcf9-041b-47ec-a371-a8d5c4a723e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:19.961846 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:20.087337 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87debcf9-041b-47ec-a371-a8d5c4a723e2 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:20.092467 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e4577d-fe52-42f3-8a16-39e13cf1a289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:20.093378 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e4577d-fe52-42f3-8a16-39e13cf1a289 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:20.093659 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e4577d-fe52-42f3-8a16-39e13cf1a289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:20.093930 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e4577d-fe52-42f3-8a16-39e13cf1a289 admin admin] PATH_INFO: `/v3/users/e856fc36a790404fa05e97eeddd7b30e` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:20.095254 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e4577d-fe52-42f3-8a16-39e13cf1a289 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e856fc36a790404fa05e97eeddd7b30e)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:20.097912 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e4577d-fe52-42f3-8a16-39e13cf1a289 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:20.213400 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-66e4577d-fe52-42f3-8a16-39e13cf1a289 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e856fc36a790404fa05e97eeddd7b30e'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:20.247578 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-66e4577d-fe52-42f3-8a16-39e13cf1a289 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e856fc36a790404fa05e97eeddd7b30e'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:20.269473 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-66e4577d-fe52-42f3-8a16-39e13cf1a289 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e856fc36a790404fa05e97eeddd7b30e'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:20.295453 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-66e4577d-fe52-42f3-8a16-39e13cf1a289 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e856fc36a790404fa05e97eeddd7b30e'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:20.300646 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-66e4577d-fe52-42f3-8a16-39e13cf1a289 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e856fc36a790404fa05e97eeddd7b30e'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:20.312553 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1091/2215] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:58:19 2022] DELETE /identity/v3/users/e856fc36a790404fa05e97eeddd7b30e => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:20.316883 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f8817-0e7d-4308-803c-2bf84873ef14 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:20.317402 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f8817-0e7d-4308-803c-2bf84873ef14 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:20.317574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f8817-0e7d-4308-803c-2bf84873ef14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:20.353423 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-ac2f8817-0e7d-4308-803c-2bf84873ef14 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:20.354775 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-ac2f8817-0e7d-4308-803c-2bf84873ef14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:20.516931 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1125/2216] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:20.539455 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e4577d-fe52-42f3-8a16-39e13cf1a289 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:20.541081 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-66e4577d-fe52-42f3-8a16-39e13cf1a289 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:20.673430 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e4577d-fe52-42f3-8a16-39e13cf1a289 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:20.686899 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158d1102-340b-4ca3-aa82-a1c649a33c0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:20.687750 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d1102-340b-4ca3-aa82-a1c649a33c0f service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:20.688123 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d1102-340b-4ca3-aa82-a1c649a33c0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:20.688803 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d1102-340b-4ca3-aa82-a1c649a33c0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:20.707638 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158d1102-340b-4ca3-aa82-a1c649a33c0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:20.711371 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158d1102-340b-4ca3-aa82-a1c649a33c0f service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:20.741577 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1092/2217] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:58:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 01:58:21.517511 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2f8817-0e7d-4308-803c-2bf84873ef14 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:21.524969 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-ac2f8817-0e7d-4308-803c-2bf84873ef14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:21.544923 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2f8817-0e7d-4308-803c-2bf84873ef14 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:21.561261 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987c0843-af40-4886-b987-17d6369dcf74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:21.562475 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c0843-af40-4886-b987-17d6369dcf74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:21.563421 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c0843-af40-4886-b987-17d6369dcf74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:21.563859 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c0843-af40-4886-b987-17d6369dcf74 admin admin] PATH_INFO: `/v3/projects/f5a9a97b5ff54aea80de0ad344b8871a` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:21.565857 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987c0843-af40-4886-b987-17d6369dcf74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5a9a97b5ff54aea80de0ad344b8871a)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:21.569874 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987c0843-af40-4886-b987-17d6369dcf74 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:21.671794 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-987c0843-af40-4886-b987-17d6369dcf74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5a9a97b5ff54aea80de0ad344b8871a', 'request_id': 'req-987c0843-af40-4886-b987-17d6369dcf74'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:21.686633 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-987c0843-af40-4886-b987-17d6369dcf74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5a9a97b5ff54aea80de0ad344b8871a', 'request_id': 'req-987c0843-af40-4886-b987-17d6369dcf74'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:21.708228 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-987c0843-af40-4886-b987-17d6369dcf74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5a9a97b5ff54aea80de0ad344b8871a', 'request_id': 'req-987c0843-af40-4886-b987-17d6369dcf74'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:21.746351 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-987c0843-af40-4886-b987-17d6369dcf74 admin admin] The token cache is being invalidate because project f5a9a97b5ff54aea80de0ad344b8871a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:58:21.746603 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-987c0843-af40-4886-b987-17d6369dcf74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:21.748379 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-987c0843-af40-4886-b987-17d6369dcf74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:21.751041 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1126/2218] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:58:21 2022] DELETE /identity/v3/projects/f5a9a97b5ff54aea80de0ad344b8871a => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:21.755027 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158d1102-340b-4ca3-aa82-a1c649a33c0f service neutron] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:21.766551 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158d1102-340b-4ca3-aa82-a1c649a33c0f service neutron] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:21.801412 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:21.875951 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc0bfb-7318-49e8-ad65-6b9f75bc56fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:21.877241 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc0bfb-7318-49e8-ad65-6b9f75bc56fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:21.877794 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc0bfb-7318-49e8-ad65-6b9f75bc56fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:21.878278 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc0bfb-7318-49e8-ad65-6b9f75bc56fd admin admin] PATH_INFO: `/v3/users/3eda796074e74c8fad4a3a8e00c1605c` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:21.879941 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdc0bfb-7318-49e8-ad65-6b9f75bc56fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=3eda796074e74c8fad4a3a8e00c1605c)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:21.883108 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdc0bfb-7318-49e8-ad65-6b9f75bc56fd admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:21.963081 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-fcdc0bfb-7318-49e8-ad65-6b9f75bc56fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3eda796074e74c8fad4a3a8e00c1605c'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:21.978558 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-fcdc0bfb-7318-49e8-ad65-6b9f75bc56fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3eda796074e74c8fad4a3a8e00c1605c'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:21.998860 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-fcdc0bfb-7318-49e8-ad65-6b9f75bc56fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3eda796074e74c8fad4a3a8e00c1605c'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:22.011925 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-fcdc0bfb-7318-49e8-ad65-6b9f75bc56fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3eda796074e74c8fad4a3a8e00c1605c'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:22.013324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-fcdc0bfb-7318-49e8-ad65-6b9f75bc56fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3eda796074e74c8fad4a3a8e00c1605c'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:22.028856 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1093/2219] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:58:21 2022] DELETE /identity/v3/users/3eda796074e74c8fad4a3a8e00c1605c => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:22.495340 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987c0843-af40-4886-b987-17d6369dcf74 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:22.497177 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-987c0843-af40-4886-b987-17d6369dcf74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:22.527955 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987c0843-af40-4886-b987-17d6369dcf74 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:22.571122 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:22.661010 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d0e19-7ae2-4219-8f1c-c06103c88c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:22.662305 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d0e19-7ae2-4219-8f1c-c06103c88c23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:22.662812 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d0e19-7ae2-4219-8f1c-c06103c88c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:22.663180 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d0e19-7ae2-4219-8f1c-c06103c88c23 admin admin] PATH_INFO: `/v3/projects/0db66ad3f2624bdba88f3aabedc7f3a4` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:22.664702 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d0e19-7ae2-4219-8f1c-c06103c88c23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0db66ad3f2624bdba88f3aabedc7f3a4)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:22.666902 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d0e19-7ae2-4219-8f1c-c06103c88c23 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:22.742149 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-de7d0e19-7ae2-4219-8f1c-c06103c88c23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0db66ad3f2624bdba88f3aabedc7f3a4', 'request_id': 'req-de7d0e19-7ae2-4219-8f1c-c06103c88c23'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:22.766678 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-de7d0e19-7ae2-4219-8f1c-c06103c88c23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0db66ad3f2624bdba88f3aabedc7f3a4', 'request_id': 'req-de7d0e19-7ae2-4219-8f1c-c06103c88c23'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:22.782462 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-de7d0e19-7ae2-4219-8f1c-c06103c88c23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0db66ad3f2624bdba88f3aabedc7f3a4', 'request_id': 'req-de7d0e19-7ae2-4219-8f1c-c06103c88c23'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:22.795003 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-de7d0e19-7ae2-4219-8f1c-c06103c88c23 admin admin] The token cache is being invalidate because project 0db66ad3f2624bdba88f3aabedc7f3a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:58:22.795446 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-de7d0e19-7ae2-4219-8f1c-c06103c88c23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:22.796219 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-de7d0e19-7ae2-4219-8f1c-c06103c88c23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:22.798784 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1127/2220] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:58:22 2022] DELETE /identity/v3/projects/0db66ad3f2624bdba88f3aabedc7f3a4 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:22.803660 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc0bfb-7318-49e8-ad65-6b9f75bc56fd admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:22.805187 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-fcdc0bfb-7318-49e8-ad65-6b9f75bc56fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:22.820311 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc0bfb-7318-49e8-ad65-6b9f75bc56fd admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:22.888020 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc7a77-fde9-449c-a302-ec2e71dc4b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:22.889083 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc7a77-fde9-449c-a302-ec2e71dc4b33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:22.889467 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc7a77-fde9-449c-a302-ec2e71dc4b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:22.889844 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc7a77-fde9-449c-a302-ec2e71dc4b33 admin admin] PATH_INFO: `/v3/users/dae643f7f0564e65947c2b1d7d5167e4` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:22.891265 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dc7a77-fde9-449c-a302-ec2e71dc4b33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dae643f7f0564e65947c2b1d7d5167e4)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:22.893370 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dc7a77-fde9-449c-a302-ec2e71dc4b33 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:22.948344 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:22.950373 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-81dc7a77-fde9-449c-a302-ec2e71dc4b33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dae643f7f0564e65947c2b1d7d5167e4'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:22.967246 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-81dc7a77-fde9-449c-a302-ec2e71dc4b33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dae643f7f0564e65947c2b1d7d5167e4'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:22.977706 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-81dc7a77-fde9-449c-a302-ec2e71dc4b33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dae643f7f0564e65947c2b1d7d5167e4'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:22.991936 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-81dc7a77-fde9-449c-a302-ec2e71dc4b33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dae643f7f0564e65947c2b1d7d5167e4'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:22.993023 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-81dc7a77-fde9-449c-a302-ec2e71dc4b33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dae643f7f0564e65947c2b1d7d5167e4'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:23.001468 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1094/2221] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:58:22 2022] DELETE /identity/v3/users/dae643f7f0564e65947c2b1d7d5167e4 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:23.259117 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba62ea-1d0e-48ad-a982-d3ac5f131f45 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:23.259833 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba62ea-1d0e-48ad-a982-d3ac5f131f45 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:23.260094 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba62ea-1d0e-48ad-a982-d3ac5f131f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:23.299563 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-37ba62ea-1d0e-48ad-a982-d3ac5f131f45 None None] MFA Rules not processed for user `f92ee0300da141d3b8a1aec100e8c0d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:23.301004 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-37ba62ea-1d0e-48ad-a982-d3ac5f131f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:23.370551 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc7a77-fde9-449c-a302-ec2e71dc4b33 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:23.372229 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-81dc7a77-fde9-449c-a302-ec2e71dc4b33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:23.397687 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1128/2222] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:23 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:23.405778 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba62ea-1d0e-48ad-a982-d3ac5f131f45 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:23.407357 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-37ba62ea-1d0e-48ad-a982-d3ac5f131f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:23.455422 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:23.465585 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc7a77-fde9-449c-a302-ec2e71dc4b33 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:23.471995 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff17f4c1-2c2d-4895-9b6b-c48b6a8dc37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:23.472816 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17f4c1-2c2d-4895-9b6b-c48b6a8dc37e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:23.473097 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17f4c1-2c2d-4895-9b6b-c48b6a8dc37e admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:23.473397 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17f4c1-2c2d-4895-9b6b-c48b6a8dc37e admin admin] PATH_INFO: `/v3/projects/1adc257f84124778aee8a0ec1360c082` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:23.474632 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff17f4c1-2c2d-4895-9b6b-c48b6a8dc37e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1adc257f84124778aee8a0ec1360c082)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:23.476548 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff17f4c1-2c2d-4895-9b6b-c48b6a8dc37e admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:23.484349 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba62ea-1d0e-48ad-a982-d3ac5f131f45 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:23.488691 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2587cbf-424d-4076-a4bb-301ec52a7b2a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c081b4cf34146ca8eb197b62a133433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:23.490083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2587cbf-424d-4076-a4bb-301ec52a7b2a service glance] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:23.490431 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2587cbf-424d-4076-a4bb-301ec52a7b2a service glance] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:23.490767 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2587cbf-424d-4076-a4bb-301ec52a7b2a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:23.492911 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-a2587cbf-424d-4076-a4bb-301ec52a7b2a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:23.531718 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-ff17f4c1-2c2d-4895-9b6b-c48b6a8dc37e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1adc257f84124778aee8a0ec1360c082', 'request_id': 'req-ff17f4c1-2c2d-4895-9b6b-c48b6a8dc37e'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:23.538932 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-ff17f4c1-2c2d-4895-9b6b-c48b6a8dc37e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1adc257f84124778aee8a0ec1360c082', 'request_id': 'req-ff17f4c1-2c2d-4895-9b6b-c48b6a8dc37e'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:23.548942 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-ff17f4c1-2c2d-4895-9b6b-c48b6a8dc37e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1adc257f84124778aee8a0ec1360c082', 'request_id': 'req-ff17f4c1-2c2d-4895-9b6b-c48b6a8dc37e'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:23.560444 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-ff17f4c1-2c2d-4895-9b6b-c48b6a8dc37e admin admin] The token cache is being invalidate because project 1adc257f84124778aee8a0ec1360c082 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:58:23.560732 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-ff17f4c1-2c2d-4895-9b6b-c48b6a8dc37e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:23.561392 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-ff17f4c1-2c2d-4895-9b6b-c48b6a8dc37e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:23.563010 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1095/2223] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:58:23 2022] DELETE /identity/v3/projects/1adc257f84124778aee8a0ec1360c082 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 01:58:23.569015 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2587cbf-424d-4076-a4bb-301ec52a7b2a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:23.569514 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132efae-eca1-4093-9074-de30d19955fc None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:23.569514 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132efae-eca1-4093-9074-de30d19955fc None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:23.569701 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132efae-eca1-4093-9074-de30d19955fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:23.571569 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2587cbf-424d-4076-a4bb-301ec52a7b2a service glance] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:23.591900 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1129/2224] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:58:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:58:23.594612 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-b132efae-eca1-4093-9074-de30d19955fc None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:23.596706 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-b132efae-eca1-4093-9074-de30d19955fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:23.677581 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:23.693808 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1096/2225] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 01:58:23.701978 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e7553-fb75-45b0-bc9c-b3e477423155 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:23.702380 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e7553-fb75-45b0-bc9c-b3e477423155 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:23.702763 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e7553-fb75-45b0-bc9c-b3e477423155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:23.722362 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-103e7553-fb75-45b0-bc9c-b3e477423155 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:23.723902 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-103e7553-fb75-45b0-bc9c-b3e477423155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:23.746643 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1130/2226] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:23.751692 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132efae-eca1-4093-9074-de30d19955fc None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:23.760422 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a9b6aa-b0a7-49af-8bc9-ff4b1020be15 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:23.760642 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a9b6aa-b0a7-49af-8bc9-ff4b1020be15 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:23.760918 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a9b6aa-b0a7-49af-8bc9-ff4b1020be15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:23.762543 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132efae-eca1-4093-9074-de30d19955fc None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:23.768183 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c6b84-9cec-4bfb-a4be-25eb2f74ba3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:23.768408 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c6b84-9cec-4bfb-a4be-25eb2f74ba3c admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:23.768609 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c6b84-9cec-4bfb-a4be-25eb2f74ba3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:23.768884 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c6b84-9cec-4bfb-a4be-25eb2f74ba3c admin admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:23.769621 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128c6b84-9cec-4bfb-a4be-25eb2f74ba3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:58:23.769925 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128c6b84-9cec-4bfb-a4be-25eb2f74ba3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:23.772835 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128c6b84-9cec-4bfb-a4be-25eb2f74ba3c admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:23.781798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-c5a9b6aa-b0a7-49af-8bc9-ff4b1020be15 None None] MFA Rules not processed for user `168e6181ad22460990f1f7361b483364`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:23.783018 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-c5a9b6aa-b0a7-49af-8bc9-ff4b1020be15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:23.788986 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1097/2227] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 01:58:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:58:23.793935 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c6b84-9cec-4bfb-a4be-25eb2f74ba3c admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:23.806942 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c6b84-9cec-4bfb-a4be-25eb2f74ba3c admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:23.814986 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c6e8f6-c7dd-44ef-b3e2-9e360ba46e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:23.817464 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c6e8f6-c7dd-44ef-b3e2-9e360ba46e69 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:23.817856 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c6e8f6-c7dd-44ef-b3e2-9e360ba46e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:23.818219 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c6e8f6-c7dd-44ef-b3e2-9e360ba46e69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:23.819037 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c6e8f6-c7dd-44ef-b3e2-9e360ba46e69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:23.821224 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c6e8f6-c7dd-44ef-b3e2-9e360ba46e69 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:23.890254 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1098/2228] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:58:23 2022] POST /identity/v3/projects => generated 407 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:58:23.894154 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c6e8f6-c7dd-44ef-b3e2-9e360ba46e69 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:23.896263 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1131/2229] 10.222.0.26 () {58 vars in 1009 bytes} [Thu Jun 23 01:58:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 01:58:23.941809 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a9b6aa-b0a7-49af-8bc9-ff4b1020be15 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:23.943876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-c5a9b6aa-b0a7-49af-8bc9-ff4b1020be15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:23.997034 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c6e8f6-c7dd-44ef-b3e2-9e360ba46e69 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:24.001722 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32506a2-8282-4033-81fc-b2f1ff95f015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:24.002405 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32506a2-8282-4033-81fc-b2f1ff95f015 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:24.002734 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32506a2-8282-4033-81fc-b2f1ff95f015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:24.003009 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32506a2-8282-4033-81fc-b2f1ff95f015 admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:24.003740 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32506a2-8282-4033-81fc-b2f1ff95f015 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:24.005666 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32506a2-8282-4033-81fc-b2f1ff95f015 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:24.019935 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a9b6aa-b0a7-49af-8bc9-ff4b1020be15 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:24.029240 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1099/2230] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:58:23 2022] POST /identity/v3/users => generated 304 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:58:24.033066 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32506a2-8282-4033-81fc-b2f1ff95f015 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:24.043539 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32506a2-8282-4033-81fc-b2f1ff95f015 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:24.048876 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeca2a4-7956-4581-b1be-9c1bb2d5210e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:24.049678 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeca2a4-7956-4581-b1be-9c1bb2d5210e admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:24.049962 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeca2a4-7956-4581-b1be-9c1bb2d5210e admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:24.050269 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeca2a4-7956-4581-b1be-9c1bb2d5210e admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:24.051010 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeca2a4-7956-4581-b1be-9c1bb2d5210e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:24.052859 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeca2a4-7956-4581-b1be-9c1bb2d5210e admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:24.053957 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:24.067434 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1100/2231] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:58:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 01:58:24.068650 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeca2a4-7956-4581-b1be-9c1bb2d5210e admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:24.078326 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeca2a4-7956-4581-b1be-9c1bb2d5210e admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:24.084334 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e152adf-221c-4002-877b-3cbc6da50432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:24.084334 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e152adf-221c-4002-877b-3cbc6da50432 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:24.084598 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e152adf-221c-4002-877b-3cbc6da50432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:24.084792 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e152adf-221c-4002-877b-3cbc6da50432 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:24.085521 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e152adf-221c-4002-877b-3cbc6da50432 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:24.087131 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e152adf-221c-4002-877b-3cbc6da50432 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:24.099005 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc57abe-7290-4051-b82c-18aaaee6c4c2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28e59688485241c4b6d51f99bbb3d059', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:24.100674 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc57abe-7290-4051-b82c-18aaaee6c4c2 service swift] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:24.101221 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc57abe-7290-4051-b82c-18aaaee6c4c2 service swift] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:24.101623 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc57abe-7290-4051-b82c-18aaaee6c4c2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:24.109237 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1101/2232] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:58:24 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:58:24.113732 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e152adf-221c-4002-877b-3cbc6da50432 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:24.129942 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e152adf-221c-4002-877b-3cbc6da50432 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:24.135248 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826e211-603d-4173-acd9-e7416bc5cfdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:24.136205 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826e211-603d-4173-acd9-e7416bc5cfdf admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:24.136551 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826e211-603d-4173-acd9-e7416bc5cfdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:24.136911 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826e211-603d-4173-acd9-e7416bc5cfdf admin admin] PATH_INFO: `/v3/projects/c2d6a32bbe5f4cad9a7c6eeaf4ddcab0/users/a67a97c99fe84d53abafec04d6197376/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:24.156979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4826e211-603d-4173-acd9-e7416bc5cfdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2d6a32bbe5f4cad9a7c6eeaf4ddcab0, user_id=a67a97c99fe84d53abafec04d6197376, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:24.161067 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4826e211-603d-4173-acd9-e7416bc5cfdf admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:24.180052 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1102/2233] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:58:24 2022] PUT /identity/v3/projects/c2d6a32bbe5f4cad9a7c6eeaf4ddcab0/users/a67a97c99fe84d53abafec04d6197376/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 01:58:24.184068 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc57abe-7290-4051-b82c-18aaaee6c4c2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:24.185917 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc57abe-7290-4051-b82c-18aaaee6c4c2 service swift] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:24.191823 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f29b1b-d76b-44f6-a55d-8d0c25d77225 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:24.192104 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f29b1b-d76b-44f6-a55d-8d0c25d77225 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:24.192433 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f29b1b-d76b-44f6-a55d-8d0c25d77225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:24.195773 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1132/2234] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Jun 23 01:58:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 23 01:58:24.227385 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-43f29b1b-d76b-44f6-a55d-8d0c25d77225 None None] MFA Rules not processed for user `a67a97c99fe84d53abafec04d6197376`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:24.229467 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-43f29b1b-d76b-44f6-a55d-8d0c25d77225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:24.287064 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:24.321918 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1103/2235] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:24 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 01:58:24.326471 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc57abe-7290-4051-b82c-18aaaee6c4c2 service swift] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:24.338067 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc57abe-7290-4051-b82c-18aaaee6c4c2 service swift] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:24.448287 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e7b427-dc9a-49af-b98b-0b41ffe3ae5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:24.449240 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e7b427-dc9a-49af-b98b-0b41ffe3ae5e admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:24.449683 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e7b427-dc9a-49af-b98b-0b41ffe3ae5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:24.450109 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e7b427-dc9a-49af-b98b-0b41ffe3ae5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:24.450976 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e7b427-dc9a-49af-b98b-0b41ffe3ae5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:24.455789 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e7b427-dc9a-49af-b98b-0b41ffe3ae5e admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:24.499581 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1133/2236] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:58:24 2022] POST /identity/v3/projects => generated 407 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 01:58:24.503580 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f29b1b-d76b-44f6-a55d-8d0c25d77225 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:24.517675 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f29b1b-d76b-44f6-a55d-8d0c25d77225 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:24.611475 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb696958-6ff7-4d2d-b5a1-17fdd830a522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:24.612694 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb696958-6ff7-4d2d-b5a1-17fdd830a522 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:24.613363 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb696958-6ff7-4d2d-b5a1-17fdd830a522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:24.613803 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb696958-6ff7-4d2d-b5a1-17fdd830a522 admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:24.614968 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb696958-6ff7-4d2d-b5a1-17fdd830a522 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:24.617119 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb696958-6ff7-4d2d-b5a1-17fdd830a522 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:24.648731 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1104/2237] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:58:24 2022] POST /identity/v3/users => generated 310 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:58:24.650106 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e7b427-dc9a-49af-b98b-0b41ffe3ae5e admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:24.699410 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e7b427-dc9a-49af-b98b-0b41ffe3ae5e admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:24.717672 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ff9641-ff13-4b77-9263-f172a4b4a512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:24.718643 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff9641-ff13-4b77-9263-f172a4b4a512 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:24.719157 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff9641-ff13-4b77-9263-f172a4b4a512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:24.720289 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff9641-ff13-4b77-9263-f172a4b4a512 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:24.720665 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ff9641-ff13-4b77-9263-f172a4b4a512 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:24.722855 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ff9641-ff13-4b77-9263-f172a4b4a512 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:24.743581 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1134/2238] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:58:24 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:58:24.745339 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb696958-6ff7-4d2d-b5a1-17fdd830a522 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:24.816460 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb696958-6ff7-4d2d-b5a1-17fdd830a522 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:24.844110 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f990b2-87f4-4e5e-983f-e42e61f73a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:24.845785 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f990b2-87f4-4e5e-983f-e42e61f73a43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:24.846073 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f990b2-87f4-4e5e-983f-e42e61f73a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:24.846487 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f990b2-87f4-4e5e-983f-e42e61f73a43 admin admin] PATH_INFO: `/v3/projects/6ee6f7dd63ad466cba70fbe315bc0549/users/5de44515ba47492c8ca88f7d33fae4fe/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:24.885589 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f990b2-87f4-4e5e-983f-e42e61f73a43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ee6f7dd63ad466cba70fbe315bc0549, user_id=5de44515ba47492c8ca88f7d33fae4fe, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:24.891793 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f990b2-87f4-4e5e-983f-e42e61f73a43 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:24.927707 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1105/2239] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:58:24 2022] PUT /identity/v3/projects/6ee6f7dd63ad466cba70fbe315bc0549/users/5de44515ba47492c8ca88f7d33fae4fe/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:24.931703 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ff9641-ff13-4b77-9263-f172a4b4a512 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:24.961426 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ff9641-ff13-4b77-9263-f172a4b4a512 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:25.050746 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:25.104190 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e29e22-e5e5-4cd0-ae1f-0228890b5332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:25.104969 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e29e22-e5e5-4cd0-ae1f-0228890b5332 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:25.105252 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e29e22-e5e5-4cd0-ae1f-0228890b5332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:25.105549 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e29e22-e5e5-4cd0-ae1f-0228890b5332 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:25.106265 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e29e22-e5e5-4cd0-ae1f-0228890b5332 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:25.108265 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e29e22-e5e5-4cd0-ae1f-0228890b5332 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:25.146362 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1135/2240] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:58:24 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:58:25.150838 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f990b2-87f4-4e5e-983f-e42e61f73a43 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:25.173751 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f990b2-87f4-4e5e-983f-e42e61f73a43 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:25.183249 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138daadb-eecf-41a0-8cea-2222e924a334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:25.183249 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138daadb-eecf-41a0-8cea-2222e924a334 admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:25.183249 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138daadb-eecf-41a0-8cea-2222e924a334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:25.183857 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138daadb-eecf-41a0-8cea-2222e924a334 admin admin] PATH_INFO: `/v3/projects/6ee6f7dd63ad466cba70fbe315bc0549/users/5de44515ba47492c8ca88f7d33fae4fe/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:25.186338 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138daadb-eecf-41a0-8cea-2222e924a334 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ee6f7dd63ad466cba70fbe315bc0549, user_id=5de44515ba47492c8ca88f7d33fae4fe, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:25.194378 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138daadb-eecf-41a0-8cea-2222e924a334 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:25.221397 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1106/2241] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:58:25 2022] PUT /identity/v3/projects/6ee6f7dd63ad466cba70fbe315bc0549/users/5de44515ba47492c8ca88f7d33fae4fe/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:25.225006 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe3480-02cc-4ce8-8176-c06b12a9b81a None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:25.225760 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe3480-02cc-4ce8-8176-c06b12a9b81a None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:25.226185 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe3480-02cc-4ce8-8176-c06b12a9b81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:25.300644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-2abe3480-02cc-4ce8-8176-c06b12a9b81a None None] MFA Rules not processed for user `5de44515ba47492c8ca88f7d33fae4fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:25.302413 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-2abe3480-02cc-4ce8-8176-c06b12a9b81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:25.508087 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1136/2242] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:25 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:25.576992 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138daadb-eecf-41a0-8cea-2222e924a334 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:25.579039 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-138daadb-eecf-41a0-8cea-2222e924a334 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:25.724552 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138daadb-eecf-41a0-8cea-2222e924a334 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:25.735932 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1bc8ff-a8cf-4130-9462-4b0a680b54db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:25.737696 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1bc8ff-a8cf-4130-9462-4b0a680b54db service cinder] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:25.738116 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1bc8ff-a8cf-4130-9462-4b0a680b54db service cinder] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:25.738701 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1bc8ff-a8cf-4130-9462-4b0a680b54db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:25.755403 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1bc8ff-a8cf-4130-9462-4b0a680b54db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:25.757821 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1bc8ff-a8cf-4130-9462-4b0a680b54db service cinder] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:25.774182 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1107/2243] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:58:25 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:58:25.999550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abe3480-02cc-4ce8-8176-c06b12a9b81a None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:26.014520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abe3480-02cc-4ce8-8176-c06b12a9b81a None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:26.018452 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247bc938-cb99-4cfc-842d-86b424566c66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:26.019102 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247bc938-cb99-4cfc-842d-86b424566c66 service cinder] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:26.019456 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247bc938-cb99-4cfc-842d-86b424566c66 service cinder] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:26.019731 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247bc938-cb99-4cfc-842d-86b424566c66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:26.125771 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247bc938-cb99-4cfc-842d-86b424566c66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:26.127910 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247bc938-cb99-4cfc-842d-86b424566c66 service cinder] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:26.142254 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1137/2244] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:58:25 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:58:28.678396 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1bc8ff-a8cf-4130-9462-4b0a680b54db service cinder] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:28.679966 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-6a1bc8ff-a8cf-4130-9462-4b0a680b54db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:28.752703 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:28.776277 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1bc8ff-a8cf-4130-9462-4b0a680b54db service cinder] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:28.781251 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0417cde5-e4ef-411e-81db-1ed03a972a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:28.782226 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0417cde5-e4ef-411e-81db-1ed03a972a3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:28.782645 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0417cde5-e4ef-411e-81db-1ed03a972a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:28.782995 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0417cde5-e4ef-411e-81db-1ed03a972a3b admin admin] PATH_INFO: `/v3/users/7e05437bac434213a4c2f7a7e437c221` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:28.784333 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0417cde5-e4ef-411e-81db-1ed03a972a3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e05437bac434213a4c2f7a7e437c221)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:28.786411 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0417cde5-e4ef-411e-81db-1ed03a972a3b admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:28.847756 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-0417cde5-e4ef-411e-81db-1ed03a972a3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e05437bac434213a4c2f7a7e437c221'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:28.863018 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-0417cde5-e4ef-411e-81db-1ed03a972a3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e05437bac434213a4c2f7a7e437c221'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:28.880615 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-0417cde5-e4ef-411e-81db-1ed03a972a3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e05437bac434213a4c2f7a7e437c221'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:28.895405 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-0417cde5-e4ef-411e-81db-1ed03a972a3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e05437bac434213a4c2f7a7e437c221'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:28.896437 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-0417cde5-e4ef-411e-81db-1ed03a972a3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e05437bac434213a4c2f7a7e437c221'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:28.906877 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1108/2245] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:58:28 2022] DELETE /identity/v3/users/7e05437bac434213a4c2f7a7e437c221 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:28.911708 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53e1bc-434d-44c4-958d-d3f90faeb33b None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:28.913589 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53e1bc-434d-44c4-958d-d3f90faeb33b None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:28.914046 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53e1bc-434d-44c4-958d-d3f90faeb33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:28.936333 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-2c53e1bc-434d-44c4-958d-d3f90faeb33b None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:28.938093 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-2c53e1bc-434d-44c4-958d-d3f90faeb33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:29.016668 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1138/2246] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 01:58:29.027389 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0417cde5-e4ef-411e-81db-1ed03a972a3b admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:29.029466 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-0417cde5-e4ef-411e-81db-1ed03a972a3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:29.135989 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0417cde5-e4ef-411e-81db-1ed03a972a3b admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:29.140524 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e17103-56dd-487a-ae18-c8f4b5704021 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:29.141368 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e17103-56dd-487a-ae18-c8f4b5704021 service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:29.141929 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e17103-56dd-487a-ae18-c8f4b5704021 service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:29.142521 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e17103-56dd-487a-ae18-c8f4b5704021 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:29.160739 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e17103-56dd-487a-ae18-c8f4b5704021 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:29.162418 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e17103-56dd-487a-ae18-c8f4b5704021 service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:29.187310 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1109/2247] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:58:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:58:29.946339 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c53e1bc-434d-44c4-958d-d3f90faeb33b None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:29.948700 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-2c53e1bc-434d-44c4-958d-d3f90faeb33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:30.006011 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c53e1bc-434d-44c4-958d-d3f90faeb33b None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:30.037597 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63bf67c-8192-43bb-9273-b14c0ca18e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:30.038478 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63bf67c-8192-43bb-9273-b14c0ca18e6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:30.038757 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63bf67c-8192-43bb-9273-b14c0ca18e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:30.039027 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63bf67c-8192-43bb-9273-b14c0ca18e6c admin admin] PATH_INFO: `/v3/projects/afaae27310d54c62ba9ac5b02db97b19` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:30.040316 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63bf67c-8192-43bb-9273-b14c0ca18e6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=afaae27310d54c62ba9ac5b02db97b19)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:30.042344 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63bf67c-8192-43bb-9273-b14c0ca18e6c admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:30.050041 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:30.176976 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-f63bf67c-8192-43bb-9273-b14c0ca18e6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afaae27310d54c62ba9ac5b02db97b19', 'request_id': 'req-f63bf67c-8192-43bb-9273-b14c0ca18e6c'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:30.186273 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-f63bf67c-8192-43bb-9273-b14c0ca18e6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afaae27310d54c62ba9ac5b02db97b19', 'request_id': 'req-f63bf67c-8192-43bb-9273-b14c0ca18e6c'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:30.201240 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-f63bf67c-8192-43bb-9273-b14c0ca18e6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afaae27310d54c62ba9ac5b02db97b19', 'request_id': 'req-f63bf67c-8192-43bb-9273-b14c0ca18e6c'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:30.216363 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-f63bf67c-8192-43bb-9273-b14c0ca18e6c admin admin] The token cache is being invalidate because project afaae27310d54c62ba9ac5b02db97b19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:58:30.216795 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-f63bf67c-8192-43bb-9273-b14c0ca18e6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:30.217742 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-f63bf67c-8192-43bb-9273-b14c0ca18e6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:30.219467 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1139/2248] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:58:29 2022] DELETE /identity/v3/projects/afaae27310d54c62ba9ac5b02db97b19 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:30.223615 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e17103-56dd-487a-ae18-c8f4b5704021 service neutron] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:30.243335 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e17103-56dd-487a-ae18-c8f4b5704021 service neutron] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:30.369984 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5111ee1d-52b9-4530-a304-26cca2b7f070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:30.371097 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111ee1d-52b9-4530-a304-26cca2b7f070 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:30.371742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111ee1d-52b9-4530-a304-26cca2b7f070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:30.372252 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111ee1d-52b9-4530-a304-26cca2b7f070 admin admin] PATH_INFO: `/v3/users/1ac106e9c3a944ee8a947557ea129143` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:30.373873 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5111ee1d-52b9-4530-a304-26cca2b7f070 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ac106e9c3a944ee8a947557ea129143)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:30.376442 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5111ee1d-52b9-4530-a304-26cca2b7f070 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:30.388413 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:30.470284 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-5111ee1d-52b9-4530-a304-26cca2b7f070 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ac106e9c3a944ee8a947557ea129143'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:30.485435 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-5111ee1d-52b9-4530-a304-26cca2b7f070 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ac106e9c3a944ee8a947557ea129143'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:30.497544 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-5111ee1d-52b9-4530-a304-26cca2b7f070 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ac106e9c3a944ee8a947557ea129143'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:30.513762 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-5111ee1d-52b9-4530-a304-26cca2b7f070 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ac106e9c3a944ee8a947557ea129143'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:30.515599 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-5111ee1d-52b9-4530-a304-26cca2b7f070 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ac106e9c3a944ee8a947557ea129143'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:30.534850 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1110/2249] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:58:30 2022] DELETE /identity/v3/users/1ac106e9c3a944ee8a947557ea129143 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:31.171483 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63bf67c-8192-43bb-9273-b14c0ca18e6c admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:31.173893 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-f63bf67c-8192-43bb-9273-b14c0ca18e6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:31.195655 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63bf67c-8192-43bb-9273-b14c0ca18e6c admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:31.329138 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea631a-1681-4125-83a8-248479880b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:31.329138 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea631a-1681-4125-83a8-248479880b90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:31.329493 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea631a-1681-4125-83a8-248479880b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:31.329690 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea631a-1681-4125-83a8-248479880b90 admin admin] PATH_INFO: `/v3/projects/a0a59bd5430d416f8976aa7aaf41834c` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:31.331010 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea631a-1681-4125-83a8-248479880b90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0a59bd5430d416f8976aa7aaf41834c)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:31.333180 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea631a-1681-4125-83a8-248479880b90 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:31.407058 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-14ea631a-1681-4125-83a8-248479880b90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0a59bd5430d416f8976aa7aaf41834c', 'request_id': 'req-14ea631a-1681-4125-83a8-248479880b90'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:31.418676 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-14ea631a-1681-4125-83a8-248479880b90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0a59bd5430d416f8976aa7aaf41834c', 'request_id': 'req-14ea631a-1681-4125-83a8-248479880b90'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:31.432883 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-14ea631a-1681-4125-83a8-248479880b90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0a59bd5430d416f8976aa7aaf41834c', 'request_id': 'req-14ea631a-1681-4125-83a8-248479880b90'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:31.445592 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-14ea631a-1681-4125-83a8-248479880b90 admin admin] The token cache is being invalidate because project a0a59bd5430d416f8976aa7aaf41834c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:58:31.445893 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-14ea631a-1681-4125-83a8-248479880b90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:31.446565 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-14ea631a-1681-4125-83a8-248479880b90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:31.451795 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1140/2250] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:58:31 2022] DELETE /identity/v3/projects/a0a59bd5430d416f8976aa7aaf41834c => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:31.453555 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2217d58d-0dfa-4571-b9f5-69e8c3b5ef42 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:31.453999 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2217d58d-0dfa-4571-b9f5-69e8c3b5ef42 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:31.454136 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2217d58d-0dfa-4571-b9f5-69e8c3b5ef42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:31.484484 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-2217d58d-0dfa-4571-b9f5-69e8c3b5ef42 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:31.487041 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-2217d58d-0dfa-4571-b9f5-69e8c3b5ef42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:31.580640 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1111/2251] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:31.581691 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92cf8c4-0d30-4341-924c-a9d838158b21 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:31.581978 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92cf8c4-0d30-4341-924c-a9d838158b21 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:31.582259 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92cf8c4-0d30-4341-924c-a9d838158b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:31.611489 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-e92cf8c4-0d30-4341-924c-a9d838158b21 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:31.612912 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-e92cf8c4-0d30-4341-924c-a9d838158b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:31.623816 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1141/2252] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:31.627894 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2217d58d-0dfa-4571-b9f5-69e8c3b5ef42 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:31.639244 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2217d58d-0dfa-4571-b9f5-69e8c3b5ef42 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:31.646928 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f432f-94f6-4889-b4c4-f7af3ca371b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:31.647840 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f432f-94f6-4889-b4c4-f7af3ca371b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:31.648931 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f432f-94f6-4889-b4c4-f7af3ca371b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:31.649340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f432f-94f6-4889-b4c4-f7af3ca371b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:31.650250 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555f432f-94f6-4889-b4c4-f7af3ca371b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:58:31.650697 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555f432f-94f6-4889-b4c4-f7af3ca371b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:31.653301 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555f432f-94f6-4889-b4c4-f7af3ca371b5 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:31.684682 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92cf8c4-0d30-4341-924c-a9d838158b21 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:31.685641 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1112/2253] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 01:58:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:58:31.694134 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92cf8c4-0d30-4341-924c-a9d838158b21 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:31.708780 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5de6a2a-86e2-4f8e-8988-925dfd9b32bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:31.710666 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de6a2a-86e2-4f8e-8988-925dfd9b32bd admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:31.711052 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de6a2a-86e2-4f8e-8988-925dfd9b32bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:31.711452 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de6a2a-86e2-4f8e-8988-925dfd9b32bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:31.712646 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5de6a2a-86e2-4f8e-8988-925dfd9b32bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:31.714643 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5de6a2a-86e2-4f8e-8988-925dfd9b32bd admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:31.726918 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:31.779407 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1142/2254] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:58:31 2022] POST /identity/v3/projects => generated 381 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 01:58:31.783118 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f432f-94f6-4889-b4c4-f7af3ca371b5 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:31.794962 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f432f-94f6-4889-b4c4-f7af3ca371b5 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:31.918175 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc60b5-f069-43dc-b0e2-057c5bc38b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:31.920111 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc60b5-f069-43dc-b0e2-057c5bc38b58 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:31.920111 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc60b5-f069-43dc-b0e2-057c5bc38b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:31.920111 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc60b5-f069-43dc-b0e2-057c5bc38b58 admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:31.920945 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cc60b5-f069-43dc-b0e2-057c5bc38b58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:31.935480 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cc60b5-f069-43dc-b0e2-057c5bc38b58 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:31.968903 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1113/2255] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:58:31 2022] POST /identity/v3/users => generated 291 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:58:31.973050 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5de6a2a-86e2-4f8e-8988-925dfd9b32bd admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:31.986460 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5de6a2a-86e2-4f8e-8988-925dfd9b32bd admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:32.007439 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd133cc2-6ecc-4f8c-a6e9-e7b0a5f74b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:32.008025 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd133cc2-6ecc-4f8c-a6e9-e7b0a5f74b6d admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:32.008305 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd133cc2-6ecc-4f8c-a6e9-e7b0a5f74b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:32.008576 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd133cc2-6ecc-4f8c-a6e9-e7b0a5f74b6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:32.009371 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd133cc2-6ecc-4f8c-a6e9-e7b0a5f74b6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:32.011010 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd133cc2-6ecc-4f8c-a6e9-e7b0a5f74b6d admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:32.040687 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1143/2256] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:58:31 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:58:32.041098 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc60b5-f069-43dc-b0e2-057c5bc38b58 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:32.048963 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:32.058609 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc60b5-f069-43dc-b0e2-057c5bc38b58 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:32.068791 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e7d8ee-7fa8-44f2-b613-be3bf0aa4f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:32.070369 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7d8ee-7fa8-44f2-b613-be3bf0aa4f35 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:32.071193 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7d8ee-7fa8-44f2-b613-be3bf0aa4f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:32.071671 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7d8ee-7fa8-44f2-b613-be3bf0aa4f35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:32.072539 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e7d8ee-7fa8-44f2-b613-be3bf0aa4f35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:32.075786 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e7d8ee-7fa8-44f2-b613-be3bf0aa4f35 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:32.102791 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1114/2257] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:58:32 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 01:58:32.107460 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd133cc2-6ecc-4f8c-a6e9-e7b0a5f74b6d admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:32.119621 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd133cc2-6ecc-4f8c-a6e9-e7b0a5f74b6d admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:32.124952 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813e2936-074c-4453-a7b1-9a62f30ab556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:32.126090 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e2936-074c-4453-a7b1-9a62f30ab556 admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:32.126451 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e2936-074c-4453-a7b1-9a62f30ab556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:32.126936 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e2936-074c-4453-a7b1-9a62f30ab556 admin admin] PATH_INFO: `/v3/projects/df3bb473462c43bb8c8148856775c28d/users/253a1a4cecd34dbba9e998c50d65bde3/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:32.148156 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813e2936-074c-4453-a7b1-9a62f30ab556 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df3bb473462c43bb8c8148856775c28d, user_id=253a1a4cecd34dbba9e998c50d65bde3, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:32.151316 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813e2936-074c-4453-a7b1-9a62f30ab556 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:32.179077 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1144/2258] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:58:32 2022] PUT /identity/v3/projects/df3bb473462c43bb8c8148856775c28d/users/253a1a4cecd34dbba9e998c50d65bde3/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:32.180475 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b3ea63-52fa-47f6-b4ca-c1716ca8dc7d None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:32.180773 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b3ea63-52fa-47f6-b4ca-c1716ca8dc7d None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:32.181062 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b3ea63-52fa-47f6-b4ca-c1716ca8dc7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:32.218306 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-48b3ea63-52fa-47f6-b4ca-c1716ca8dc7d None None] MFA Rules not processed for user `253a1a4cecd34dbba9e998c50d65bde3`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:32.222296 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-48b3ea63-52fa-47f6-b4ca-c1716ca8dc7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:32.351451 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1115/2259] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:32.355971 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813e2936-074c-4453-a7b1-9a62f30ab556 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:32.435682 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813e2936-074c-4453-a7b1-9a62f30ab556 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:32.444659 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267aff02-e225-4c3e-ac78-bfd609f93e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:32.445591 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267aff02-e225-4c3e-ac78-bfd609f93e21 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:32.446052 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267aff02-e225-4c3e-ac78-bfd609f93e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:32.446527 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267aff02-e225-4c3e-ac78-bfd609f93e21 admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:32.447648 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267aff02-e225-4c3e-ac78-bfd609f93e21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:32.449968 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267aff02-e225-4c3e-ac78-bfd609f93e21 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:32.483347 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1145/2260] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:58:32 2022] POST /identity/v3/projects => generated 381 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 01:58:32.487753 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b3ea63-52fa-47f6-b4ca-c1716ca8dc7d None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:32.512728 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b3ea63-52fa-47f6-b4ca-c1716ca8dc7d None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:32.660002 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea40a860-fbc1-4e9a-bea9-1529cc6df009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:32.660766 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea40a860-fbc1-4e9a-bea9-1529cc6df009 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:32.661076 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea40a860-fbc1-4e9a-bea9-1529cc6df009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:32.661892 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea40a860-fbc1-4e9a-bea9-1529cc6df009 admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:32.677985 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea40a860-fbc1-4e9a-bea9-1529cc6df009 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:32.680165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea40a860-fbc1-4e9a-bea9-1529cc6df009 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:32.734252 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1116/2261] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:58:32 2022] POST /identity/v3/users => generated 297 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:58:32.735647 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267aff02-e225-4c3e-ac78-bfd609f93e21 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:32.747099 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267aff02-e225-4c3e-ac78-bfd609f93e21 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:32.752606 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c518e-98d7-4c1e-bcdf-f378284c08a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:32.753374 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c518e-98d7-4c1e-bcdf-f378284c08a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:32.753760 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c518e-98d7-4c1e-bcdf-f378284c08a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:32.754274 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c518e-98d7-4c1e-bcdf-f378284c08a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:32.756286 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2c518e-98d7-4c1e-bcdf-f378284c08a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:32.759000 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2c518e-98d7-4c1e-bcdf-f378284c08a7 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:32.808019 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1146/2262] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:58:32 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 01:58:32.812884 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea40a860-fbc1-4e9a-bea9-1529cc6df009 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:32.825090 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea40a860-fbc1-4e9a-bea9-1529cc6df009 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:32.831156 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0490d41-c3fd-4283-9440-323cfb6ff5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:32.832952 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0490d41-c3fd-4283-9440-323cfb6ff5fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:32.833389 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0490d41-c3fd-4283-9440-323cfb6ff5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:32.833757 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0490d41-c3fd-4283-9440-323cfb6ff5fb admin admin] PATH_INFO: `/v3/projects/6d75b5d445aa4a38a1ed21edf77467b1/users/98351d562adf407ebd2eabf40fe45aae/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:32.853469 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0490d41-c3fd-4283-9440-323cfb6ff5fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d75b5d445aa4a38a1ed21edf77467b1, user_id=98351d562adf407ebd2eabf40fe45aae, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:32.857057 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0490d41-c3fd-4283-9440-323cfb6ff5fb admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:32.879628 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1117/2263] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:58:32 2022] PUT /identity/v3/projects/6d75b5d445aa4a38a1ed21edf77467b1/users/98351d562adf407ebd2eabf40fe45aae/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:32.883604 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c518e-98d7-4c1e-bcdf-f378284c08a7 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:32.895818 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c518e-98d7-4c1e-bcdf-f378284c08a7 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:33.015256 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ff04ac-20c6-43ea-aaf7-1651649727f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:33.016201 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff04ac-20c6-43ea-aaf7-1651649727f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:33.016633 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff04ac-20c6-43ea-aaf7-1651649727f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:33.017072 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff04ac-20c6-43ea-aaf7-1651649727f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:33.018409 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ff04ac-20c6-43ea-aaf7-1651649727f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:33.020112 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ff04ac-20c6-43ea-aaf7-1651649727f2 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:33.033491 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1147/2264] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:58:32 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:58:33.039744 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0490d41-c3fd-4283-9440-323cfb6ff5fb admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:33.053566 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0490d41-c3fd-4283-9440-323cfb6ff5fb admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:33.059020 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca5116f-bc4e-4ec0-a701-01402b2fe0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:33.060574 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca5116f-bc4e-4ec0-a701-01402b2fe0ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:33.061013 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca5116f-bc4e-4ec0-a701-01402b2fe0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:33.061442 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca5116f-bc4e-4ec0-a701-01402b2fe0ca admin admin] PATH_INFO: `/v3/projects/6d75b5d445aa4a38a1ed21edf77467b1/users/98351d562adf407ebd2eabf40fe45aae/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:33.064571 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca5116f-bc4e-4ec0-a701-01402b2fe0ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d75b5d445aa4a38a1ed21edf77467b1, user_id=98351d562adf407ebd2eabf40fe45aae, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:33.067825 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca5116f-bc4e-4ec0-a701-01402b2fe0ca admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:33.081914 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1118/2265] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:58:33 2022] PUT /identity/v3/projects/6d75b5d445aa4a38a1ed21edf77467b1/users/98351d562adf407ebd2eabf40fe45aae/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:33.093133 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a7831d-a82e-4085-ab9e-ab146ac2b181 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:33.093419 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a7831d-a82e-4085-ab9e-ab146ac2b181 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:33.093695 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a7831d-a82e-4085-ab9e-ab146ac2b181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:33.133143 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-e1a7831d-a82e-4085-ab9e-ab146ac2b181 None None] MFA Rules not processed for user `98351d562adf407ebd2eabf40fe45aae`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:33.134368 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-e1a7831d-a82e-4085-ab9e-ab146ac2b181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:33.148728 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:33.219566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1148/2266] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:33 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:33.230894 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca5116f-bc4e-4ec0-a701-01402b2fe0ca admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:33.232717 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-4ca5116f-bc4e-4ec0-a701-01402b2fe0ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:33.289754 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:33.315947 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca5116f-bc4e-4ec0-a701-01402b2fe0ca admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:33.319726 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb2da4e-b5f4-445e-bb79-a73e570477f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:33.320449 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2da4e-b5f4-445e-bb79-a73e570477f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:33.320819 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2da4e-b5f4-445e-bb79-a73e570477f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:33.321174 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2da4e-b5f4-445e-bb79-a73e570477f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:33.332194 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb2da4e-b5f4-445e-bb79-a73e570477f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:33.333728 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb2da4e-b5f4-445e-bb79-a73e570477f7 service cinder] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:33.357294 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1119/2267] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:58:33 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:58:33.828903 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a7831d-a82e-4085-ab9e-ab146ac2b181 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:33.901984 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a7831d-a82e-4085-ab9e-ab146ac2b181 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:33.906766 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d19542-1204-4774-a0d7-c419dd61132b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:33.907670 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d19542-1204-4774-a0d7-c419dd61132b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:33.907947 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d19542-1204-4774-a0d7-c419dd61132b admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:33.908217 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d19542-1204-4774-a0d7-c419dd61132b admin admin] PATH_INFO: `/v3/users/253a1a4cecd34dbba9e998c50d65bde3` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:33.910826 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d19542-1204-4774-a0d7-c419dd61132b admin admin] RBAC: Authorizing `identity:delete_user(user_id=253a1a4cecd34dbba9e998c50d65bde3)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:33.912847 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d19542-1204-4774-a0d7-c419dd61132b admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:33.970879 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-f4d19542-1204-4774-a0d7-c419dd61132b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '253a1a4cecd34dbba9e998c50d65bde3'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:33.987048 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-f4d19542-1204-4774-a0d7-c419dd61132b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '253a1a4cecd34dbba9e998c50d65bde3'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:33.997088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-f4d19542-1204-4774-a0d7-c419dd61132b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '253a1a4cecd34dbba9e998c50d65bde3'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:34.009718 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-f4d19542-1204-4774-a0d7-c419dd61132b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '253a1a4cecd34dbba9e998c50d65bde3'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:34.010884 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-f4d19542-1204-4774-a0d7-c419dd61132b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '253a1a4cecd34dbba9e998c50d65bde3'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:34.017462 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1149/2268] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:58:33 2022] DELETE /identity/v3/users/253a1a4cecd34dbba9e998c50d65bde3 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:34.024689 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3cee2-b2c7-475a-88f3-d703fcbc1ed9 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:34.025292 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3cee2-b2c7-475a-88f3-d703fcbc1ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:34.025292 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3cee2-b2c7-475a-88f3-d703fcbc1ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:34.055941 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-07f3cee2-b2c7-475a-88f3-d703fcbc1ed9 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:34.056995 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-07f3cee2-b2c7-475a-88f3-d703fcbc1ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:34.194962 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1120/2269] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:34.205443 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d19542-1204-4774-a0d7-c419dd61132b admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:34.208980 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-f4d19542-1204-4774-a0d7-c419dd61132b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:34.254421 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:34.282816 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d19542-1204-4774-a0d7-c419dd61132b admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:34.286357 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5039a45-e109-4fac-ad31-abf04f2268d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:34.286996 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5039a45-e109-4fac-ad31-abf04f2268d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:34.287306 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5039a45-e109-4fac-ad31-abf04f2268d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:34.287578 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5039a45-e109-4fac-ad31-abf04f2268d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:34.306511 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5039a45-e109-4fac-ad31-abf04f2268d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:34.308089 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5039a45-e109-4fac-ad31-abf04f2268d6 service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:34.319331 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1150/2270] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:58:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 01:58:34.660689 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f3cee2-b2c7-475a-88f3-d703fcbc1ed9 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:34.666739 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-07f3cee2-b2c7-475a-88f3-d703fcbc1ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:34.699423 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f3cee2-b2c7-475a-88f3-d703fcbc1ed9 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:34.706246 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1e707e-c698-4fe9-a002-47471d093368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:34.707277 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e707e-c698-4fe9-a002-47471d093368 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:34.707407 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e707e-c698-4fe9-a002-47471d093368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:34.707680 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e707e-c698-4fe9-a002-47471d093368 admin admin] PATH_INFO: `/v3/projects/df3bb473462c43bb8c8148856775c28d` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:34.709044 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1e707e-c698-4fe9-a002-47471d093368 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df3bb473462c43bb8c8148856775c28d)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:34.714863 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1e707e-c698-4fe9-a002-47471d093368 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:34.778318 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-4f1e707e-c698-4fe9-a002-47471d093368 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df3bb473462c43bb8c8148856775c28d', 'request_id': 'req-4f1e707e-c698-4fe9-a002-47471d093368'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:34.784978 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-4f1e707e-c698-4fe9-a002-47471d093368 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df3bb473462c43bb8c8148856775c28d', 'request_id': 'req-4f1e707e-c698-4fe9-a002-47471d093368'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:34.795577 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-4f1e707e-c698-4fe9-a002-47471d093368 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df3bb473462c43bb8c8148856775c28d', 'request_id': 'req-4f1e707e-c698-4fe9-a002-47471d093368'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:34.807733 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-4f1e707e-c698-4fe9-a002-47471d093368 admin admin] The token cache is being invalidate because project df3bb473462c43bb8c8148856775c28d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:58:34.808163 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-4f1e707e-c698-4fe9-a002-47471d093368 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:34.808808 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-4f1e707e-c698-4fe9-a002-47471d093368 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:34.810376 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1121/2271] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:58:34 2022] DELETE /identity/v3/projects/df3bb473462c43bb8c8148856775c28d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:34.814600 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5039a45-e109-4fac-ad31-abf04f2268d6 service neutron] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:34.824646 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5039a45-e109-4fac-ad31-abf04f2268d6 service neutron] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:34.889740 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcf9741-2ebe-4f9a-b1a8-bbf825b9fb47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:34.890637 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf9741-2ebe-4f9a-b1a8-bbf825b9fb47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:34.890946 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf9741-2ebe-4f9a-b1a8-bbf825b9fb47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:34.891246 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf9741-2ebe-4f9a-b1a8-bbf825b9fb47 admin admin] PATH_INFO: `/v3/users/98351d562adf407ebd2eabf40fe45aae` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:34.892481 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcf9741-2ebe-4f9a-b1a8-bbf825b9fb47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98351d562adf407ebd2eabf40fe45aae)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:34.894488 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcf9741-2ebe-4f9a-b1a8-bbf825b9fb47 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:34.953481 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-dfcf9741-2ebe-4f9a-b1a8-bbf825b9fb47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98351d562adf407ebd2eabf40fe45aae'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:34.968069 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-dfcf9741-2ebe-4f9a-b1a8-bbf825b9fb47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98351d562adf407ebd2eabf40fe45aae'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:34.981792 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-dfcf9741-2ebe-4f9a-b1a8-bbf825b9fb47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98351d562adf407ebd2eabf40fe45aae'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:34.995590 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-dfcf9741-2ebe-4f9a-b1a8-bbf825b9fb47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98351d562adf407ebd2eabf40fe45aae'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:34.996382 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-dfcf9741-2ebe-4f9a-b1a8-bbf825b9fb47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98351d562adf407ebd2eabf40fe45aae'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:35.005747 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1151/2272] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:58:34 2022] DELETE /identity/v3/users/98351d562adf407ebd2eabf40fe45aae => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:35.346287 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1e707e-c698-4fe9-a002-47471d093368 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:35.347940 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-4f1e707e-c698-4fe9-a002-47471d093368 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:35.363453 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1e707e-c698-4fe9-a002-47471d093368 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:35.438055 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f4739e-9120-450d-a64b-eb70f30b672d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:35.439693 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4739e-9120-450d-a64b-eb70f30b672d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:35.439990 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4739e-9120-450d-a64b-eb70f30b672d admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:35.440902 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4739e-9120-450d-a64b-eb70f30b672d admin admin] PATH_INFO: `/v3/projects/6d75b5d445aa4a38a1ed21edf77467b1` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:35.442556 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f4739e-9120-450d-a64b-eb70f30b672d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d75b5d445aa4a38a1ed21edf77467b1)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:35.445049 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f4739e-9120-450d-a64b-eb70f30b672d admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:35.447958 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:35.531163 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-d6f4739e-9120-450d-a64b-eb70f30b672d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d75b5d445aa4a38a1ed21edf77467b1', 'request_id': 'req-d6f4739e-9120-450d-a64b-eb70f30b672d'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:35.542322 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-d6f4739e-9120-450d-a64b-eb70f30b672d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d75b5d445aa4a38a1ed21edf77467b1', 'request_id': 'req-d6f4739e-9120-450d-a64b-eb70f30b672d'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:35.560410 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-d6f4739e-9120-450d-a64b-eb70f30b672d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d75b5d445aa4a38a1ed21edf77467b1', 'request_id': 'req-d6f4739e-9120-450d-a64b-eb70f30b672d'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:35.582222 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-d6f4739e-9120-450d-a64b-eb70f30b672d admin admin] The token cache is being invalidate because project 6d75b5d445aa4a38a1ed21edf77467b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:58:35.582600 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-d6f4739e-9120-450d-a64b-eb70f30b672d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:35.583626 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-d6f4739e-9120-450d-a64b-eb70f30b672d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:35.585963 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1122/2273] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:58:35 2022] DELETE /identity/v3/projects/6d75b5d445aa4a38a1ed21edf77467b1 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:35.591700 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba6019-8115-428f-b40d-2d9533e91651 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:35.593645 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba6019-8115-428f-b40d-2d9533e91651 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:35.594088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba6019-8115-428f-b40d-2d9533e91651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:35.615532 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-37ba6019-8115-428f-b40d-2d9533e91651 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:35.617023 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-37ba6019-8115-428f-b40d-2d9533e91651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:35.674970 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:35.701174 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1152/2274] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:35.707425 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b989a1d-792a-4baf-baaa-24eb19c7b4f5 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:35.707721 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b989a1d-792a-4baf-baaa-24eb19c7b4f5 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:35.708035 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b989a1d-792a-4baf-baaa-24eb19c7b4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:35.755341 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-7b989a1d-792a-4baf-baaa-24eb19c7b4f5 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:35.755341 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-7b989a1d-792a-4baf-baaa-24eb19c7b4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:35.789812 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1123/2275] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:35.803243 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba6019-8115-428f-b40d-2d9533e91651 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:35.839472 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba6019-8115-428f-b40d-2d9533e91651 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:35.844392 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabfc227-458c-4c50-b050-bb92ee009f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:35.845250 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabfc227-458c-4c50-b050-bb92ee009f04 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:35.845591 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabfc227-458c-4c50-b050-bb92ee009f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:35.845979 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabfc227-458c-4c50-b050-bb92ee009f04 admin admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:35.846793 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabfc227-458c-4c50-b050-bb92ee009f04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:58:35.847854 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabfc227-458c-4c50-b050-bb92ee009f04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:35.855254 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabfc227-458c-4c50-b050-bb92ee009f04 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:35.874042 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1153/2276] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 01:58:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:58:35.878552 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b989a1d-792a-4baf-baaa-24eb19c7b4f5 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:35.889516 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b989a1d-792a-4baf-baaa-24eb19c7b4f5 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:35.894656 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15c4b21-cfec-477a-96a9-d34b4e8585b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:35.895379 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c4b21-cfec-477a-96a9-d34b4e8585b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:35.895670 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c4b21-cfec-477a-96a9-d34b4e8585b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:35.895961 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c4b21-cfec-477a-96a9-d34b4e8585b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:35.896662 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15c4b21-cfec-477a-96a9-d34b4e8585b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:35.898617 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15c4b21-cfec-477a-96a9-d34b4e8585b2 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:35.935894 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1124/2277] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:58:35 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 01:58:35.939657 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabfc227-458c-4c50-b050-bb92ee009f04 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:35.953309 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabfc227-458c-4c50-b050-bb92ee009f04 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:36.039047 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ea19e-f4b0-4e9c-832e-bb3accb4cb60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:36.039970 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ea19e-f4b0-4e9c-832e-bb3accb4cb60 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:36.040359 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ea19e-f4b0-4e9c-832e-bb3accb4cb60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:36.040710 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ea19e-f4b0-4e9c-832e-bb3accb4cb60 admin admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:36.041671 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1ea19e-f4b0-4e9c-832e-bb3accb4cb60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:36.044486 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1ea19e-f4b0-4e9c-832e-bb3accb4cb60 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:36.068196 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1154/2278] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:58:35 2022] POST /identity/v3/users => generated 287 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:58:36.072398 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15c4b21-cfec-477a-96a9-d34b4e8585b2 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:36.084560 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15c4b21-cfec-477a-96a9-d34b4e8585b2 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:36.090065 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f1fa7-8d51-4feb-b292-bb00742544f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:36.090774 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f1fa7-8d51-4feb-b292-bb00742544f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:36.091057 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f1fa7-8d51-4feb-b292-bb00742544f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:36.091375 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f1fa7-8d51-4feb-b292-bb00742544f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:36.092121 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18f1fa7-8d51-4feb-b292-bb00742544f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:36.093737 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18f1fa7-8d51-4feb-b292-bb00742544f7 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:36.109758 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1125/2279] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:58:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:58:36.113618 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ea19e-f4b0-4e9c-832e-bb3accb4cb60 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:36.125667 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ea19e-f4b0-4e9c-832e-bb3accb4cb60 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:36.130695 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd1caf3-2039-4f20-b10e-873291b6d035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:36.131704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1caf3-2039-4f20-b10e-873291b6d035 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:36.131985 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1caf3-2039-4f20-b10e-873291b6d035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:36.132261 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1caf3-2039-4f20-b10e-873291b6d035 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:36.133005 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd1caf3-2039-4f20-b10e-873291b6d035 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:36.134816 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd1caf3-2039-4f20-b10e-873291b6d035 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:36.149032 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1155/2280] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:58:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:58:36.153346 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f1fa7-8d51-4feb-b292-bb00742544f7 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:36.166273 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f1fa7-8d51-4feb-b292-bb00742544f7 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:36.171200 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c320df83-71c7-4775-91a6-41258b825439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:36.172146 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c320df83-71c7-4775-91a6-41258b825439 admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:36.172733 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c320df83-71c7-4775-91a6-41258b825439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:36.173013 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c320df83-71c7-4775-91a6-41258b825439 admin admin] PATH_INFO: `/v3/projects/897052f7bdb24e96bdeb986788a74623/users/9415cedeb11d4ea29bbc33071e50c82d/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:36.192457 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c320df83-71c7-4775-91a6-41258b825439 admin admin] RBAC: Authorizing `identity:create_grant(project_id=897052f7bdb24e96bdeb986788a74623, user_id=9415cedeb11d4ea29bbc33071e50c82d, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:36.195123 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c320df83-71c7-4775-91a6-41258b825439 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:36.212477 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1126/2281] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:58:36 2022] PUT /identity/v3/projects/897052f7bdb24e96bdeb986788a74623/users/9415cedeb11d4ea29bbc33071e50c82d/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 01:58:36.217827 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b56e08-902f-451a-bf78-6edd3bbfa766 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:36.218151 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b56e08-902f-451a-bf78-6edd3bbfa766 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:36.218429 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b56e08-902f-451a-bf78-6edd3bbfa766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:36.237676 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-80b56e08-902f-451a-bf78-6edd3bbfa766 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:36.238908 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-80b56e08-902f-451a-bf78-6edd3bbfa766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:36.324042 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1156/2282] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:36.330743 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c320df83-71c7-4775-91a6-41258b825439 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:36.334219 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-c320df83-71c7-4775-91a6-41258b825439 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:36.416333 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c320df83-71c7-4775-91a6-41258b825439 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:36.420690 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7cb2d-4ce3-4649-8fe9-8f61fa2a6055 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:36.421588 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7cb2d-4ce3-4649-8fe9-8f61fa2a6055 service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:36.421967 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7cb2d-4ce3-4649-8fe9-8f61fa2a6055 service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:36.422336 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7cb2d-4ce3-4649-8fe9-8f61fa2a6055 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:36.438981 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7cb2d-4ce3-4649-8fe9-8f61fa2a6055 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:36.440620 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7cb2d-4ce3-4649-8fe9-8f61fa2a6055 service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:36.451953 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1127/2283] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:58:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 01:58:36.554019 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b56e08-902f-451a-bf78-6edd3bbfa766 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:36.556165 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-80b56e08-902f-451a-bf78-6edd3bbfa766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:36.572240 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b56e08-902f-451a-bf78-6edd3bbfa766 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:36.585536 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce03c80d-821e-4338-bf26-c36106662a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:36.587490 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03c80d-821e-4338-bf26-c36106662a06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:36.587951 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03c80d-821e-4338-bf26-c36106662a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:36.588383 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03c80d-821e-4338-bf26-c36106662a06 admin admin] PATH_INFO: `/v3/users/a67a97c99fe84d53abafec04d6197376` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:36.589888 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce03c80d-821e-4338-bf26-c36106662a06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a67a97c99fe84d53abafec04d6197376)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:36.592065 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce03c80d-821e-4338-bf26-c36106662a06 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:36.664303 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-ce03c80d-821e-4338-bf26-c36106662a06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a67a97c99fe84d53abafec04d6197376'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:36.671359 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:36.702671 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-ce03c80d-821e-4338-bf26-c36106662a06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a67a97c99fe84d53abafec04d6197376'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:36.718110 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-ce03c80d-821e-4338-bf26-c36106662a06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a67a97c99fe84d53abafec04d6197376'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:36.730945 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-ce03c80d-821e-4338-bf26-c36106662a06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a67a97c99fe84d53abafec04d6197376'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:36.731731 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-ce03c80d-821e-4338-bf26-c36106662a06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a67a97c99fe84d53abafec04d6197376'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:36.738735 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1157/2284] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:58:36 2022] DELETE /identity/v3/users/a67a97c99fe84d53abafec04d6197376 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:36.743089 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5fa4ef-bc0d-4686-9a28-0258a0f66d19 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:36.743566 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5fa4ef-bc0d-4686-9a28-0258a0f66d19 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:36.744013 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5fa4ef-bc0d-4686-9a28-0258a0f66d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:36.763503 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-5d5fa4ef-bc0d-4686-9a28-0258a0f66d19 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:36.764679 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-5d5fa4ef-bc0d-4686-9a28-0258a0f66d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:36.768615 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd5d83-2432-4834-bfaa-07dea84268b6 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:36.769093 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd5d83-2432-4834-bfaa-07dea84268b6 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:36.769373 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd5d83-2432-4834-bfaa-07dea84268b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:36.794037 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-96fd5d83-2432-4834-bfaa-07dea84268b6 None None] MFA Rules not processed for user `3d72a03c4eec4412856ee48cb967c46d`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:36.795407 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:36.796882 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-96fd5d83-2432-4834-bfaa-07dea84268b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:36.846361 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1128/2285] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:36.857739 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5fa4ef-bc0d-4686-9a28-0258a0f66d19 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:36.859343 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-5d5fa4ef-bc0d-4686-9a28-0258a0f66d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:36.909151 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1158/2286] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:36.934538 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5fa4ef-bc0d-4686-9a28-0258a0f66d19 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:36.938469 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e62e42-64ab-46b0-a180-02a090857589 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:36.939129 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e62e42-64ab-46b0-a180-02a090857589 service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:36.939746 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e62e42-64ab-46b0-a180-02a090857589 service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:36.939956 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e62e42-64ab-46b0-a180-02a090857589 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:36.950899 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e62e42-64ab-46b0-a180-02a090857589 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:36.952540 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e62e42-64ab-46b0-a180-02a090857589 service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:36.963277 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1129/2287] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:58:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:58:37.724039 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fd5d83-2432-4834-bfaa-07dea84268b6 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:37.726398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-96fd5d83-2432-4834-bfaa-07dea84268b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:37.752444 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fd5d83-2432-4834-bfaa-07dea84268b6 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:37.758899 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f1b54-82c7-4e22-972f-4765ea3ded6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:37.760101 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f1b54-82c7-4e22-972f-4765ea3ded6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:37.760483 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f1b54-82c7-4e22-972f-4765ea3ded6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:37.760873 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f1b54-82c7-4e22-972f-4765ea3ded6d admin admin] PATH_INFO: `/v3/projects/c2d6a32bbe5f4cad9a7c6eeaf4ddcab0` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:37.762249 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16f1b54-82c7-4e22-972f-4765ea3ded6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2d6a32bbe5f4cad9a7c6eeaf4ddcab0)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:37.764458 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16f1b54-82c7-4e22-972f-4765ea3ded6d admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:37.819178 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180af77b-ca07-458e-9124-df65fa2581d0 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:37.819808 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180af77b-ca07-458e-9124-df65fa2581d0 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:37.819808 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180af77b-ca07-458e-9124-df65fa2581d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:37.850210 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-a16f1b54-82c7-4e22-972f-4765ea3ded6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2d6a32bbe5f4cad9a7c6eeaf4ddcab0', 'request_id': 'req-a16f1b54-82c7-4e22-972f-4765ea3ded6d'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:37.860139 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-a16f1b54-82c7-4e22-972f-4765ea3ded6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2d6a32bbe5f4cad9a7c6eeaf4ddcab0', 'request_id': 'req-a16f1b54-82c7-4e22-972f-4765ea3ded6d'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:37.872955 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-a16f1b54-82c7-4e22-972f-4765ea3ded6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2d6a32bbe5f4cad9a7c6eeaf4ddcab0', 'request_id': 'req-a16f1b54-82c7-4e22-972f-4765ea3ded6d'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:37.893021 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-a16f1b54-82c7-4e22-972f-4765ea3ded6d admin admin] The token cache is being invalidate because project c2d6a32bbe5f4cad9a7c6eeaf4ddcab0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:58:37.893553 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-a16f1b54-82c7-4e22-972f-4765ea3ded6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:37.894583 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-a16f1b54-82c7-4e22-972f-4765ea3ded6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:37.897322 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1159/2288] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:58:37 2022] DELETE /identity/v3/projects/c2d6a32bbe5f4cad9a7c6eeaf4ddcab0 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:37.901041 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f1b54-82c7-4e22-972f-4765ea3ded6d admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:37.902771 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-a16f1b54-82c7-4e22-972f-4765ea3ded6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:37.905531 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-180af77b-ca07-458e-9124-df65fa2581d0 None None] MFA Rules not processed for user `9415cedeb11d4ea29bbc33071e50c82d`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:37.909085 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-180af77b-ca07-458e-9124-df65fa2581d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:37.996369 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f1b54-82c7-4e22-972f-4765ea3ded6d admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:38.004848 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb446fd7-4a0b-429f-919c-c7a512ff340e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:38.006735 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb446fd7-4a0b-429f-919c-c7a512ff340e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:38.007028 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb446fd7-4a0b-429f-919c-c7a512ff340e admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:38.007510 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb446fd7-4a0b-429f-919c-c7a512ff340e admin admin] PATH_INFO: `/v3/users/5de44515ba47492c8ca88f7d33fae4fe` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:38.009106 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb446fd7-4a0b-429f-919c-c7a512ff340e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5de44515ba47492c8ca88f7d33fae4fe)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:38.011124 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb446fd7-4a0b-429f-919c-c7a512ff340e admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:38.013610 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:38.026875 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1130/2289] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:38.031492 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180af77b-ca07-458e-9124-df65fa2581d0 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:38.033624 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-180af77b-ca07-458e-9124-df65fa2581d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:38.055201 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180af77b-ca07-458e-9124-df65fa2581d0 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:38.061799 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576a392d-e307-4b79-97ed-3a0d1516ddde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:38.062659 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a392d-e307-4b79-97ed-3a0d1516ddde admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:38.062940 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a392d-e307-4b79-97ed-3a0d1516ddde admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:38.063326 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a392d-e307-4b79-97ed-3a0d1516ddde admin admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:38.064367 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576a392d-e307-4b79-97ed-3a0d1516ddde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:38.066805 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576a392d-e307-4b79-97ed-3a0d1516ddde admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:38.100932 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-fb446fd7-4a0b-429f-919c-c7a512ff340e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5de44515ba47492c8ca88f7d33fae4fe'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:38.130592 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-fb446fd7-4a0b-429f-919c-c7a512ff340e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5de44515ba47492c8ca88f7d33fae4fe'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:38.132506 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1131/2290] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:58:38 2022] POST /identity/v3/projects => generated 373 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 01:58:38.136560 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576a392d-e307-4b79-97ed-3a0d1516ddde admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:38.145453 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-fb446fd7-4a0b-429f-919c-c7a512ff340e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5de44515ba47492c8ca88f7d33fae4fe'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:38.155437 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-fb446fd7-4a0b-429f-919c-c7a512ff340e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5de44515ba47492c8ca88f7d33fae4fe'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:38.156180 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-fb446fd7-4a0b-429f-919c-c7a512ff340e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5de44515ba47492c8ca88f7d33fae4fe'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:38.162936 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1160/2291] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:58:37 2022] DELETE /identity/v3/users/5de44515ba47492c8ca88f7d33fae4fe => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 01:58:38.233001 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:38.277373 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576a392d-e307-4b79-97ed-3a0d1516ddde admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:38.414067 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222133c8-d6ef-47fb-ad66-f9f63e191d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:38.414777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222133c8-d6ef-47fb-ad66-f9f63e191d3c admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:38.416697 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222133c8-d6ef-47fb-ad66-f9f63e191d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:38.417079 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222133c8-d6ef-47fb-ad66-f9f63e191d3c admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:38.417900 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222133c8-d6ef-47fb-ad66-f9f63e191d3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:38.428694 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222133c8-d6ef-47fb-ad66-f9f63e191d3c admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:38.496282 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb446fd7-4a0b-429f-919c-c7a512ff340e admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:38.498021 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-fb446fd7-4a0b-429f-919c-c7a512ff340e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:38.506385 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1132/2292] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:58:38 2022] POST /identity/v3/users => generated 293 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 01:58:38.522931 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb446fd7-4a0b-429f-919c-c7a512ff340e admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:38.547850 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0b35d3-73f8-46c5-849f-c45fa9670cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:38.548618 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0b35d3-73f8-46c5-849f-c45fa9670cfa admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:38.549011 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0b35d3-73f8-46c5-849f-c45fa9670cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:38.549359 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0b35d3-73f8-46c5-849f-c45fa9670cfa admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:38.550182 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0b35d3-73f8-46c5-849f-c45fa9670cfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:38.552132 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0b35d3-73f8-46c5-849f-c45fa9670cfa admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:38.565430 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1161/2293] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:58:38 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 01:58:38.569528 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222133c8-d6ef-47fb-ad66-f9f63e191d3c admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:38.581307 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222133c8-d6ef-47fb-ad66-f9f63e191d3c admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:38.587601 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ec8f1-ce46-44b7-98f7-ee53bce19382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:38.588784 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ec8f1-ce46-44b7-98f7-ee53bce19382 admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:38.589199 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ec8f1-ce46-44b7-98f7-ee53bce19382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:38.589570 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ec8f1-ce46-44b7-98f7-ee53bce19382 admin admin] PATH_INFO: `/v3/projects/1cf85d202f4c41139c99e12310aec5d2/users/9587ce8f79bd42c2a3ae7ae52d4e49a5/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:38.619843 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8ec8f1-ce46-44b7-98f7-ee53bce19382 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cf85d202f4c41139c99e12310aec5d2, user_id=9587ce8f79bd42c2a3ae7ae52d4e49a5, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:38.622771 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8ec8f1-ce46-44b7-98f7-ee53bce19382 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:38.648834 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1133/2294] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:58:38 2022] PUT /identity/v3/projects/1cf85d202f4c41139c99e12310aec5d2/users/9587ce8f79bd42c2a3ae7ae52d4e49a5/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 01:58:38.653112 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0b35d3-73f8-46c5-849f-c45fa9670cfa admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:38.671740 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0b35d3-73f8-46c5-849f-c45fa9670cfa admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:38.805054 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121a4185-0fb7-45db-a951-a0675017cb35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:38.805725 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a4185-0fb7-45db-a951-a0675017cb35 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:38.806005 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a4185-0fb7-45db-a951-a0675017cb35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:38.806278 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a4185-0fb7-45db-a951-a0675017cb35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:38.807011 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121a4185-0fb7-45db-a951-a0675017cb35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:38.808833 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121a4185-0fb7-45db-a951-a0675017cb35 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:38.826401 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1162/2295] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:58:38 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:58:38.831005 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ec8f1-ce46-44b7-98f7-ee53bce19382 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:38.851381 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ec8f1-ce46-44b7-98f7-ee53bce19382 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:38.876104 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37fa371-472d-45b0-9027-46ad0016b44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:38.877414 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37fa371-472d-45b0-9027-46ad0016b44d admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:38.877594 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37fa371-472d-45b0-9027-46ad0016b44d admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:38.877875 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37fa371-472d-45b0-9027-46ad0016b44d admin admin] PATH_INFO: `/v3/projects/1cf85d202f4c41139c99e12310aec5d2/users/9587ce8f79bd42c2a3ae7ae52d4e49a5/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:38.883522 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37fa371-472d-45b0-9027-46ad0016b44d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cf85d202f4c41139c99e12310aec5d2, user_id=9587ce8f79bd42c2a3ae7ae52d4e49a5, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:38.890961 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37fa371-472d-45b0-9027-46ad0016b44d admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:38.909218 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1134/2296] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:58:38 2022] PUT /identity/v3/projects/1cf85d202f4c41139c99e12310aec5d2/users/9587ce8f79bd42c2a3ae7ae52d4e49a5/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:39.099576 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121a4185-0fb7-45db-a951-a0675017cb35 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:39.110311 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-121a4185-0fb7-45db-a951-a0675017cb35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:39.121729 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121a4185-0fb7-45db-a951-a0675017cb35 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:39.210245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443abc58-29d1-4df2-a846-bdbeecb2cd5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:39.211199 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443abc58-29d1-4df2-a846-bdbeecb2cd5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:39.212118 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443abc58-29d1-4df2-a846-bdbeecb2cd5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:39.212461 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443abc58-29d1-4df2-a846-bdbeecb2cd5c admin admin] PATH_INFO: `/v3/projects/6ee6f7dd63ad466cba70fbe315bc0549` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:39.213806 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443abc58-29d1-4df2-a846-bdbeecb2cd5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ee6f7dd63ad466cba70fbe315bc0549)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:39.215843 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443abc58-29d1-4df2-a846-bdbeecb2cd5c admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:39.226558 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:39.312288 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-443abc58-29d1-4df2-a846-bdbeecb2cd5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ee6f7dd63ad466cba70fbe315bc0549', 'request_id': 'req-443abc58-29d1-4df2-a846-bdbeecb2cd5c'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:39.322317 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-443abc58-29d1-4df2-a846-bdbeecb2cd5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ee6f7dd63ad466cba70fbe315bc0549', 'request_id': 'req-443abc58-29d1-4df2-a846-bdbeecb2cd5c'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:39.333459 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-443abc58-29d1-4df2-a846-bdbeecb2cd5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ee6f7dd63ad466cba70fbe315bc0549', 'request_id': 'req-443abc58-29d1-4df2-a846-bdbeecb2cd5c'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:39.352045 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-443abc58-29d1-4df2-a846-bdbeecb2cd5c admin admin] The token cache is being invalidate because project 6ee6f7dd63ad466cba70fbe315bc0549 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:58:39.352333 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-443abc58-29d1-4df2-a846-bdbeecb2cd5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:39.353326 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-443abc58-29d1-4df2-a846-bdbeecb2cd5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:39.364025 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1163/2297] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:58:39 2022] DELETE /identity/v3/projects/6ee6f7dd63ad466cba70fbe315bc0549 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:39.365999 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174caa95-5d58-43fd-8311-919898e519d3 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:39.366289 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174caa95-5d58-43fd-8311-919898e519d3 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:39.366568 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174caa95-5d58-43fd-8311-919898e519d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:39.385691 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-174caa95-5d58-43fd-8311-919898e519d3 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:39.387045 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-174caa95-5d58-43fd-8311-919898e519d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:39.462818 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1135/2298] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:39.470800 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b1929-7be8-4897-8c17-52f157555651 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:39.471281 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b1929-7be8-4897-8c17-52f157555651 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:39.471630 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b1929-7be8-4897-8c17-52f157555651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:39.492531 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-847b1929-7be8-4897-8c17-52f157555651 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:39.493852 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-847b1929-7be8-4897-8c17-52f157555651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:39.505598 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1164/2299] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:39.507073 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174caa95-5d58-43fd-8311-919898e519d3 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:39.518257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174caa95-5d58-43fd-8311-919898e519d3 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:39.524900 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89787f3-619f-43d9-a3e0-7a441ac9e7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:39.525562 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89787f3-619f-43d9-a3e0-7a441ac9e7bf admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:39.525915 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89787f3-619f-43d9-a3e0-7a441ac9e7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:39.526335 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89787f3-619f-43d9-a3e0-7a441ac9e7bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:39.527849 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89787f3-619f-43d9-a3e0-7a441ac9e7bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:58:39.528493 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89787f3-619f-43d9-a3e0-7a441ac9e7bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:39.530632 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89787f3-619f-43d9-a3e0-7a441ac9e7bf admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:39.544413 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1136/2300] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 01:58:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 01:58:39.554283 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b1929-7be8-4897-8c17-52f157555651 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:39.566518 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b1929-7be8-4897-8c17-52f157555651 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:39.573488 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e09578-e75d-41b1-8eae-689501c3ca02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:39.575646 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e09578-e75d-41b1-8eae-689501c3ca02 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:39.575989 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e09578-e75d-41b1-8eae-689501c3ca02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:39.576442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e09578-e75d-41b1-8eae-689501c3ca02 admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:39.577878 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e09578-e75d-41b1-8eae-689501c3ca02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:39.581169 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e09578-e75d-41b1-8eae-689501c3ca02 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:39.618435 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1165/2301] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:58:39 2022] POST /identity/v3/projects => generated 375 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:58:39.622612 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89787f3-619f-43d9-a3e0-7a441ac9e7bf admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:39.633281 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89787f3-619f-43d9-a3e0-7a441ac9e7bf admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:39.668848 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:39.706586 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4d9c3f-6b6e-4744-b350-ec7e1a094d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:39.707301 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4d9c3f-6b6e-4744-b350-ec7e1a094d99 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:39.707589 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4d9c3f-6b6e-4744-b350-ec7e1a094d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:39.707868 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4d9c3f-6b6e-4744-b350-ec7e1a094d99 admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:39.708627 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4d9c3f-6b6e-4744-b350-ec7e1a094d99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:39.710653 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4d9c3f-6b6e-4744-b350-ec7e1a094d99 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:39.740744 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e09578-e75d-41b1-8eae-689501c3ca02 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:39.743063 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1137/2302] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:58:39 2022] POST /identity/v3/users => generated 288 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 01:58:39.753775 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e09578-e75d-41b1-8eae-689501c3ca02 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:39.758820 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e1c533-daa5-46bf-b3f2-6da2585a40e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:39.761879 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e1c533-daa5-46bf-b3f2-6da2585a40e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:39.762220 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e1c533-daa5-46bf-b3f2-6da2585a40e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:39.762511 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e1c533-daa5-46bf-b3f2-6da2585a40e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:39.763343 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e1c533-daa5-46bf-b3f2-6da2585a40e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:39.765224 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e1c533-daa5-46bf-b3f2-6da2585a40e0 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:39.777038 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1166/2303] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:58:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:58:39.781369 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4d9c3f-6b6e-4744-b350-ec7e1a094d99 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:39.794988 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4d9c3f-6b6e-4744-b350-ec7e1a094d99 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:39.800952 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dfdc56-dbcf-4f1d-bdab-43b1830d4f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:39.801934 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfdc56-dbcf-4f1d-bdab-43b1830d4f30 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:39.802453 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfdc56-dbcf-4f1d-bdab-43b1830d4f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:39.802806 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfdc56-dbcf-4f1d-bdab-43b1830d4f30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:39.803869 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dfdc56-dbcf-4f1d-bdab-43b1830d4f30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:39.805748 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dfdc56-dbcf-4f1d-bdab-43b1830d4f30 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:39.821137 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1138/2304] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:58:39 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 01:58:39.829199 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e1c533-daa5-46bf-b3f2-6da2585a40e0 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:39.843907 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e1c533-daa5-46bf-b3f2-6da2585a40e0 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:39.850166 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21edbb6f-86ab-4f74-a127-c9da8540a67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:39.851820 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21edbb6f-86ab-4f74-a127-c9da8540a67a admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:39.851820 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21edbb6f-86ab-4f74-a127-c9da8540a67a admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:39.851820 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21edbb6f-86ab-4f74-a127-c9da8540a67a admin admin] PATH_INFO: `/v3/projects/2f60d868caa5418b8bbc3732e8cdb29a/users/20b6a0c09693435a982721fd0b494a7c/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:39.871424 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21edbb6f-86ab-4f74-a127-c9da8540a67a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f60d868caa5418b8bbc3732e8cdb29a, user_id=20b6a0c09693435a982721fd0b494a7c, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:39.874603 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21edbb6f-86ab-4f74-a127-c9da8540a67a admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:39.897906 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1167/2305] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:58:39 2022] PUT /identity/v3/projects/2f60d868caa5418b8bbc3732e8cdb29a/users/20b6a0c09693435a982721fd0b494a7c/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:39.910920 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb956b-66dc-49d5-a5de-91b67056cc8e None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:39.911479 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb956b-66dc-49d5-a5de-91b67056cc8e None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:39.911736 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb956b-66dc-49d5-a5de-91b67056cc8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:39.945662 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-7eeb956b-66dc-49d5-a5de-91b67056cc8e None None] MFA Rules not processed for user `20b6a0c09693435a982721fd0b494a7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:39.947505 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-7eeb956b-66dc-49d5-a5de-91b67056cc8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:39.996518 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708021b-c11c-4f85-af33-facd87ea9bd8 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:39.996909 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708021b-c11c-4f85-af33-facd87ea9bd8 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:39.997272 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708021b-c11c-4f85-af33-facd87ea9bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:40.041795 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-1708021b-c11c-4f85-af33-facd87ea9bd8 None None] MFA Rules not processed for user `9587ce8f79bd42c2a3ae7ae52d4e49a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:40.045030 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-1708021b-c11c-4f85-af33-facd87ea9bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:40.086557 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1139/2306] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:40.098641 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeb956b-66dc-49d5-a5de-91b67056cc8e None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:40.100689 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-7eeb956b-66dc-49d5-a5de-91b67056cc8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:40.240263 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1168/2307] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:39 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:40.243441 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251b0c0f-6dc9-4df9-b019-cea232f0f4c6 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:40.244140 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251b0c0f-6dc9-4df9-b019-cea232f0f4c6 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:40.244868 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251b0c0f-6dc9-4df9-b019-cea232f0f4c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:40.263571 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeb956b-66dc-49d5-a5de-91b67056cc8e None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:40.273049 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4b9b19-da51-48b3-8054-d90ccff069a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:40.273726 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b9b19-da51-48b3-8054-d90ccff069a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:40.274008 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b9b19-da51-48b3-8054-d90ccff069a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:40.274285 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b9b19-da51-48b3-8054-d90ccff069a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:40.282870 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-251b0c0f-6dc9-4df9-b019-cea232f0f4c6 None None] MFA Rules not processed for user `168e6181ad22460990f1f7361b483364`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:40.284000 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-251b0c0f-6dc9-4df9-b019-cea232f0f4c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:40.294784 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4b9b19-da51-48b3-8054-d90ccff069a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:40.294784 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4b9b19-da51-48b3-8054-d90ccff069a8 service cinder] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:40.305153 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1140/2308] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:58:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:58:40.305662 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:40.306369 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4b9b19-da51-48b3-8054-d90ccff069a8 service cinder] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:40.308082 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-3e4b9b19-da51-48b3-8054-d90ccff069a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:40.324755 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4b9b19-da51-48b3-8054-d90ccff069a8 service cinder] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:40.331117 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6909827e-bb12-4bd1-82a5-5877ac170beb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:40.332340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909827e-bb12-4bd1-82a5-5877ac170beb service cinder] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:40.332744 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909827e-bb12-4bd1-82a5-5877ac170beb service cinder] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:40.333387 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909827e-bb12-4bd1-82a5-5877ac170beb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:40.335763 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-6909827e-bb12-4bd1-82a5-5877ac170beb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:40.399947 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1169/2309] 10.222.0.26 () {58 vars in 1009 bytes} [Thu Jun 23 01:58:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:40.405575 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:40.413358 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251b0c0f-6dc9-4df9-b019-cea232f0f4c6 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:40.414524 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-251b0c0f-6dc9-4df9-b019-cea232f0f4c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:40.457156 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6909827e-bb12-4bd1-82a5-5877ac170beb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:40.462142 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6909827e-bb12-4bd1-82a5-5877ac170beb service cinder] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:40.473507 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1141/2310] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:58:40 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:58:40.565381 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251b0c0f-6dc9-4df9-b019-cea232f0f4c6 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:40.572460 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd81e0e7-af6d-4c29-8bec-bce3cdbaa67c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28e59688485241c4b6d51f99bbb3d059', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:40.573203 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81e0e7-af6d-4c29-8bec-bce3cdbaa67c service swift] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:40.573695 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81e0e7-af6d-4c29-8bec-bce3cdbaa67c service swift] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:40.574150 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81e0e7-af6d-4c29-8bec-bce3cdbaa67c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:40.585613 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd81e0e7-af6d-4c29-8bec-bce3cdbaa67c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:40.587518 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd81e0e7-af6d-4c29-8bec-bce3cdbaa67c service swift] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:40.611638 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1170/2311] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Jun 23 01:58:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 01:58:42.103545 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6909827e-bb12-4bd1-82a5-5877ac170beb service cinder] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:42.116437 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6909827e-bb12-4bd1-82a5-5877ac170beb service cinder] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:42.120274 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c405e7cf-c875-47e0-a3dd-c6037efa61a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:42.121071 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c405e7cf-c875-47e0-a3dd-c6037efa61a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:42.121184 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c405e7cf-c875-47e0-a3dd-c6037efa61a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:42.121408 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c405e7cf-c875-47e0-a3dd-c6037efa61a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:42.132186 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c405e7cf-c875-47e0-a3dd-c6037efa61a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:42.133958 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c405e7cf-c875-47e0-a3dd-c6037efa61a0 service cinder] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:42.159985 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1142/2312] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:58:42 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:58:50.594776 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=50 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:52.141175 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=16 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:52.755479 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd81e0e7-af6d-4c29-8bec-bce3cdbaa67c service swift] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:52.837867 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd81e0e7-af6d-4c29-8bec-bce3cdbaa67c service swift] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:52.844509 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf051e-1f16-4194-9e33-3d0c4c67d86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:52.845778 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf051e-1f16-4194-9e33-3d0c4c67d86c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:52.846061 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf051e-1f16-4194-9e33-3d0c4c67d86c admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:52.846339 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf051e-1f16-4194-9e33-3d0c4c67d86c admin admin] PATH_INFO: `/v3/users/20b6a0c09693435a982721fd0b494a7c` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:52.848764 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcf051e-1f16-4194-9e33-3d0c4c67d86c admin admin] RBAC: Authorizing `identity:delete_user(user_id=20b6a0c09693435a982721fd0b494a7c)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:52.852499 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcf051e-1f16-4194-9e33-3d0c4c67d86c admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:52.945833 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-abcf051e-1f16-4194-9e33-3d0c4c67d86c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20b6a0c09693435a982721fd0b494a7c'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:52.974735 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-abcf051e-1f16-4194-9e33-3d0c4c67d86c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20b6a0c09693435a982721fd0b494a7c'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:52.985996 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-abcf051e-1f16-4194-9e33-3d0c4c67d86c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20b6a0c09693435a982721fd0b494a7c'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:52.996693 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-abcf051e-1f16-4194-9e33-3d0c4c67d86c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20b6a0c09693435a982721fd0b494a7c'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:52.997902 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-abcf051e-1f16-4194-9e33-3d0c4c67d86c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20b6a0c09693435a982721fd0b494a7c'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:53.008037 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1171/2313] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:58:52 2022] DELETE /identity/v3/users/20b6a0c09693435a982721fd0b494a7c => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:53.012389 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc11a23d-1c67-4cd3-90d8-d50970f2cb86 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:53.012681 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc11a23d-1c67-4cd3-90d8-d50970f2cb86 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:53.012964 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc11a23d-1c67-4cd3-90d8-d50970f2cb86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:53.038693 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-fc11a23d-1c67-4cd3-90d8-d50970f2cb86 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:53.039977 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-fc11a23d-1c67-4cd3-90d8-d50970f2cb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:53.160190 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1143/2314] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:53.169801 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf051e-1f16-4194-9e33-3d0c4c67d86c admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:53.171518 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-abcf051e-1f16-4194-9e33-3d0c4c67d86c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:53.295765 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf051e-1f16-4194-9e33-3d0c4c67d86c admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:53.300095 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf01ea40-d8e0-4dc1-a75e-5924a40553fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:53.300755 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01ea40-d8e0-4dc1-a75e-5924a40553fa service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:53.301038 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01ea40-d8e0-4dc1-a75e-5924a40553fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:53.303307 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01ea40-d8e0-4dc1-a75e-5924a40553fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:53.316690 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf01ea40-d8e0-4dc1-a75e-5924a40553fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:53.318993 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf01ea40-d8e0-4dc1-a75e-5924a40553fa service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:53.329992 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1172/2315] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:58:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:58:53.947240 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc11a23d-1c67-4cd3-90d8-d50970f2cb86 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:53.949705 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-fc11a23d-1c67-4cd3-90d8-d50970f2cb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:53.965198 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc11a23d-1c67-4cd3-90d8-d50970f2cb86 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:53.978362 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355ace10-3d92-4b69-814f-b533d96dfb91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:53.979496 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ace10-3d92-4b69-814f-b533d96dfb91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:53.979775 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ace10-3d92-4b69-814f-b533d96dfb91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:53.980224 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ace10-3d92-4b69-814f-b533d96dfb91 admin admin] PATH_INFO: `/v3/projects/2f60d868caa5418b8bbc3732e8cdb29a` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:53.981495 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355ace10-3d92-4b69-814f-b533d96dfb91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f60d868caa5418b8bbc3732e8cdb29a)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:53.983481 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355ace10-3d92-4b69-814f-b533d96dfb91 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:54.036601 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-355ace10-3d92-4b69-814f-b533d96dfb91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f60d868caa5418b8bbc3732e8cdb29a', 'request_id': 'req-355ace10-3d92-4b69-814f-b533d96dfb91'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:54.043948 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-355ace10-3d92-4b69-814f-b533d96dfb91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f60d868caa5418b8bbc3732e8cdb29a', 'request_id': 'req-355ace10-3d92-4b69-814f-b533d96dfb91'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:54.054148 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-355ace10-3d92-4b69-814f-b533d96dfb91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f60d868caa5418b8bbc3732e8cdb29a', 'request_id': 'req-355ace10-3d92-4b69-814f-b533d96dfb91'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:54.068769 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-355ace10-3d92-4b69-814f-b533d96dfb91 admin admin] The token cache is being invalidate because project 2f60d868caa5418b8bbc3732e8cdb29a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:58:54.069162 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-355ace10-3d92-4b69-814f-b533d96dfb91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:54.069878 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-355ace10-3d92-4b69-814f-b533d96dfb91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:54.071968 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1144/2316] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:58:53 2022] DELETE /identity/v3/projects/2f60d868caa5418b8bbc3732e8cdb29a => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:54.076999 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f7fec8-1988-4291-9fe9-2709d9337e5e None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:54.077527 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f7fec8-1988-4291-9fe9-2709d9337e5e None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:54.077527 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f7fec8-1988-4291-9fe9-2709d9337e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:54.096549 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-48f7fec8-1988-4291-9fe9-2709d9337e5e None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:54.097802 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-48f7fec8-1988-4291-9fe9-2709d9337e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:54.117160 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:54.166871 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1173/2317] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:54.175750 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76051290-71ed-4490-a7d3-a3509261e975 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:54.176238 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76051290-71ed-4490-a7d3-a3509261e975 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:54.176493 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76051290-71ed-4490-a7d3-a3509261e975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:54.201752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-76051290-71ed-4490-a7d3-a3509261e975 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:54.203070 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-76051290-71ed-4490-a7d3-a3509261e975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:54.214198 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1145/2318] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:54.218894 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f7fec8-1988-4291-9fe9-2709d9337e5e None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:54.236261 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f7fec8-1988-4291-9fe9-2709d9337e5e None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:54.245172 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96adbd0c-dbc2-4a7b-900a-47c93f94ab1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:54.245172 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96adbd0c-dbc2-4a7b-900a-47c93f94ab1a admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:54.245172 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96adbd0c-dbc2-4a7b-900a-47c93f94ab1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:54.245172 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96adbd0c-dbc2-4a7b-900a-47c93f94ab1a admin admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:54.247235 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96adbd0c-dbc2-4a7b-900a-47c93f94ab1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:58:54.247235 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96adbd0c-dbc2-4a7b-900a-47c93f94ab1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:54.247235 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96adbd0c-dbc2-4a7b-900a-47c93f94ab1a admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:54.262301 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1174/2319] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 01:58:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:58:54.263574 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76051290-71ed-4490-a7d3-a3509261e975 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:54.275606 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76051290-71ed-4490-a7d3-a3509261e975 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:54.281674 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f48c3cd-854f-472d-96be-31d3e0a05b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:54.283410 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f48c3cd-854f-472d-96be-31d3e0a05b45 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:54.283788 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f48c3cd-854f-472d-96be-31d3e0a05b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:54.284189 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f48c3cd-854f-472d-96be-31d3e0a05b45 admin admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:54.285129 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f48c3cd-854f-472d-96be-31d3e0a05b45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:54.287684 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f48c3cd-854f-472d-96be-31d3e0a05b45 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:54.328042 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1146/2320] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:58:54 2022] POST /identity/v3/projects => generated 379 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:58:54.332307 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96adbd0c-dbc2-4a7b-900a-47c93f94ab1a admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:54.347504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96adbd0c-dbc2-4a7b-900a-47c93f94ab1a admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:54.431020 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734146ea-5501-42e7-820c-de2732e5b4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:54.431889 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734146ea-5501-42e7-820c-de2732e5b4f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:54.432238 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734146ea-5501-42e7-820c-de2732e5b4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:54.432573 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734146ea-5501-42e7-820c-de2732e5b4f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:54.433332 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734146ea-5501-42e7-820c-de2732e5b4f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:54.435850 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734146ea-5501-42e7-820c-de2732e5b4f4 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:54.475265 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1175/2321] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:58:54 2022] POST /identity/v3/users => generated 290 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:58:54.476860 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f48c3cd-854f-472d-96be-31d3e0a05b45 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:54.493929 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f48c3cd-854f-472d-96be-31d3e0a05b45 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:54.500416 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93daac5d-7f5f-4603-8721-0bb7a18f7bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:54.501422 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93daac5d-7f5f-4603-8721-0bb7a18f7bcb admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:54.501877 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93daac5d-7f5f-4603-8721-0bb7a18f7bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:54.502225 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93daac5d-7f5f-4603-8721-0bb7a18f7bcb admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:54.504076 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93daac5d-7f5f-4603-8721-0bb7a18f7bcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:54.506766 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93daac5d-7f5f-4603-8721-0bb7a18f7bcb admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:54.527361 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1147/2322] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:58:54 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:58:54.528863 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734146ea-5501-42e7-820c-de2732e5b4f4 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:54.546846 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734146ea-5501-42e7-820c-de2732e5b4f4 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:54.552756 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2614d43a-b666-49b0-b40e-995cac685d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:54.553539 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614d43a-b666-49b0-b40e-995cac685d52 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:54.553922 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614d43a-b666-49b0-b40e-995cac685d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:54.554494 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614d43a-b666-49b0-b40e-995cac685d52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:54.555445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2614d43a-b666-49b0-b40e-995cac685d52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:54.557196 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2614d43a-b666-49b0-b40e-995cac685d52 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:54.570395 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1176/2323] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:58:54 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:58:54.573450 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93daac5d-7f5f-4603-8721-0bb7a18f7bcb admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:54.586334 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93daac5d-7f5f-4603-8721-0bb7a18f7bcb admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:54.593109 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62210a-5987-4d19-86ad-846364aca09d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:54.594162 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62210a-5987-4d19-86ad-846364aca09d admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:54.594539 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62210a-5987-4d19-86ad-846364aca09d admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:54.594913 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62210a-5987-4d19-86ad-846364aca09d admin admin] PATH_INFO: `/v3/projects/59bb100f7d034605bb875abd1a0775ec/users/c14a81624a264ab280d46d063ae676ac/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:54.650045 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62210a-5987-4d19-86ad-846364aca09d admin admin] RBAC: Authorizing `identity:create_grant(project_id=59bb100f7d034605bb875abd1a0775ec, user_id=c14a81624a264ab280d46d063ae676ac, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:54.653002 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62210a-5987-4d19-86ad-846364aca09d admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:54.684373 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1148/2324] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:58:54 2022] PUT /identity/v3/projects/59bb100f7d034605bb875abd1a0775ec/users/c14a81624a264ab280d46d063ae676ac/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:54.689853 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d16476-dd61-484b-a828-aecbb5b246d0 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:54.690465 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d16476-dd61-484b-a828-aecbb5b246d0 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:54.690465 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d16476-dd61-484b-a828-aecbb5b246d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:54.722688 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-b0d16476-dd61-484b-a828-aecbb5b246d0 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:54.724020 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-b0d16476-dd61-484b-a828-aecbb5b246d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:54.828685 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1177/2325] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:54.836633 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62210a-5987-4d19-86ad-846364aca09d admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:54.839532 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-4e62210a-5987-4d19-86ad-846364aca09d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:54.869664 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:54.942017 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62210a-5987-4d19-86ad-846364aca09d admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:54.946499 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ceaaa7-f8d1-4a71-9b20-0d530b348b6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:54.948784 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ceaaa7-f8d1-4a71-9b20-0d530b348b6e service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:54.949993 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ceaaa7-f8d1-4a71-9b20-0d530b348b6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:54.950367 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ceaaa7-f8d1-4a71-9b20-0d530b348b6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:54.965660 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ceaaa7-f8d1-4a71-9b20-0d530b348b6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:54.967436 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ceaaa7-f8d1-4a71-9b20-0d530b348b6e service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:54.982772 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1149/2326] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:58:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:58:55.011235 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3fc604-8fb5-4a68-9412-eb801a23835d None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:55.011566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3fc604-8fb5-4a68-9412-eb801a23835d None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:55.011845 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3fc604-8fb5-4a68-9412-eb801a23835d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:55.020957 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8deb87f-8ca7-4114-9b55-1d8da6aa7cbf None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:55.021652 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8deb87f-8ca7-4114-9b55-1d8da6aa7cbf None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:55.022150 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8deb87f-8ca7-4114-9b55-1d8da6aa7cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:55.044313 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-3d3fc604-8fb5-4a68-9412-eb801a23835d None None] MFA Rules not processed for user `168e6181ad22460990f1f7361b483364`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:55.045599 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-3d3fc604-8fb5-4a68-9412-eb801a23835d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:55.048326 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-d8deb87f-8ca7-4114-9b55-1d8da6aa7cbf None None] MFA Rules not processed for user `0f92d1dfc6c74b51aec7212f9a244a0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:58:55.049611 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-d8deb87f-8ca7-4114-9b55-1d8da6aa7cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:55.092944 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:58:55.157899 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1150/2327] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:58:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:55.159688 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1178/2328] 10.222.0.26 () {58 vars in 1009 bytes} [Thu Jun 23 01:58:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:58:55.175656 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8deb87f-8ca7-4114-9b55-1d8da6aa7cbf None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:55.177413 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-d8deb87f-8ca7-4114-9b55-1d8da6aa7cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:55.284039 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8deb87f-8ca7-4114-9b55-1d8da6aa7cbf None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:55.301999 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce9dfb7-3518-491e-86d7-c9c0514a894d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28e59688485241c4b6d51f99bbb3d059', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:55.302963 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce9dfb7-3518-491e-86d7-c9c0514a894d service swift] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:55.303585 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce9dfb7-3518-491e-86d7-c9c0514a894d service swift] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:55.304058 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce9dfb7-3518-491e-86d7-c9c0514a894d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:55.315618 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce9dfb7-3518-491e-86d7-c9c0514a894d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:55.317450 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce9dfb7-3518-491e-86d7-c9c0514a894d service swift] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:55.330745 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1151/2329] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Jun 23 01:58:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 23 01:58:59.043608 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3fc604-8fb5-4a68-9412-eb801a23835d None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:59.045393 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-3d3fc604-8fb5-4a68-9412-eb801a23835d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:58:59.073121 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3fc604-8fb5-4a68-9412-eb801a23835d None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:58:59.089837 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c94347-a945-4175-92d5-44fdcdccbc5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:58:59.108314 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c94347-a945-4175-92d5-44fdcdccbc5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:58:59.108791 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c94347-a945-4175-92d5-44fdcdccbc5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:58:59.112592 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c94347-a945-4175-92d5-44fdcdccbc5b admin admin] PATH_INFO: `/v3/users/9415cedeb11d4ea29bbc33071e50c82d` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:58:59.115471 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c94347-a945-4175-92d5-44fdcdccbc5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9415cedeb11d4ea29bbc33071e50c82d)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:58:59.117950 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c94347-a945-4175-92d5-44fdcdccbc5b admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:58:59.211962 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-49c94347-a945-4175-92d5-44fdcdccbc5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9415cedeb11d4ea29bbc33071e50c82d'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:59.234767 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-49c94347-a945-4175-92d5-44fdcdccbc5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9415cedeb11d4ea29bbc33071e50c82d'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:59.251997 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-49c94347-a945-4175-92d5-44fdcdccbc5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9415cedeb11d4ea29bbc33071e50c82d'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:59.271493 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-49c94347-a945-4175-92d5-44fdcdccbc5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9415cedeb11d4ea29bbc33071e50c82d'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:59.272519 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-49c94347-a945-4175-92d5-44fdcdccbc5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9415cedeb11d4ea29bbc33071e50c82d'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:58:59.301257 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1179/2330] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:58:59 2022] DELETE /identity/v3/users/9415cedeb11d4ea29bbc33071e50c82d => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:58:59.947737 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce9dfb7-3518-491e-86d7-c9c0514a894d service swift] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:58:59.977687 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce9dfb7-3518-491e-86d7-c9c0514a894d service swift] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:00.212699 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:59:00.244015 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd16603-4a51-428d-9de0-c08bb67b6fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:00.244912 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd16603-4a51-428d-9de0-c08bb67b6fbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:00.245197 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd16603-4a51-428d-9de0-c08bb67b6fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:00.245478 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd16603-4a51-428d-9de0-c08bb67b6fbb admin admin] PATH_INFO: `/v3/projects/897052f7bdb24e96bdeb986788a74623` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:00.250932 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd16603-4a51-428d-9de0-c08bb67b6fbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=897052f7bdb24e96bdeb986788a74623)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:00.253394 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd16603-4a51-428d-9de0-c08bb67b6fbb admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:00.373101 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-8cd16603-4a51-428d-9de0-c08bb67b6fbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '897052f7bdb24e96bdeb986788a74623', 'request_id': 'req-8cd16603-4a51-428d-9de0-c08bb67b6fbb'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:00.404884 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-8cd16603-4a51-428d-9de0-c08bb67b6fbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '897052f7bdb24e96bdeb986788a74623', 'request_id': 'req-8cd16603-4a51-428d-9de0-c08bb67b6fbb'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:00.429835 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-8cd16603-4a51-428d-9de0-c08bb67b6fbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '897052f7bdb24e96bdeb986788a74623', 'request_id': 'req-8cd16603-4a51-428d-9de0-c08bb67b6fbb'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:00.449713 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-8cd16603-4a51-428d-9de0-c08bb67b6fbb admin admin] The token cache is being invalidate because project 897052f7bdb24e96bdeb986788a74623 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:59:00.450011 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-8cd16603-4a51-428d-9de0-c08bb67b6fbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:00.458495 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-8cd16603-4a51-428d-9de0-c08bb67b6fbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:00.460273 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1152/2331] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:58:59 2022] DELETE /identity/v3/projects/897052f7bdb24e96bdeb986788a74623 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:59:00.465393 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c94347-a945-4175-92d5-44fdcdccbc5b admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:00.470137 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-49c94347-a945-4175-92d5-44fdcdccbc5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:00.514787 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c94347-a945-4175-92d5-44fdcdccbc5b admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:00.693065 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77204321-7c0a-4a0d-8fab-c59659645e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:00.694105 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77204321-7c0a-4a0d-8fab-c59659645e72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:00.694394 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77204321-7c0a-4a0d-8fab-c59659645e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:00.694675 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77204321-7c0a-4a0d-8fab-c59659645e72 admin admin] PATH_INFO: `/v3/users/9587ce8f79bd42c2a3ae7ae52d4e49a5` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:00.698575 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77204321-7c0a-4a0d-8fab-c59659645e72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9587ce8f79bd42c2a3ae7ae52d4e49a5)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:00.701318 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77204321-7c0a-4a0d-8fab-c59659645e72 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:00.822862 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-77204321-7c0a-4a0d-8fab-c59659645e72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9587ce8f79bd42c2a3ae7ae52d4e49a5'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:00.824230 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:59:00.855384 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-77204321-7c0a-4a0d-8fab-c59659645e72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9587ce8f79bd42c2a3ae7ae52d4e49a5'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:00.871056 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-77204321-7c0a-4a0d-8fab-c59659645e72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9587ce8f79bd42c2a3ae7ae52d4e49a5'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:00.883344 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-77204321-7c0a-4a0d-8fab-c59659645e72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9587ce8f79bd42c2a3ae7ae52d4e49a5'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:00.884255 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-77204321-7c0a-4a0d-8fab-c59659645e72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9587ce8f79bd42c2a3ae7ae52d4e49a5'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:00.895929 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1180/2332] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:59:00 2022] DELETE /identity/v3/users/9587ce8f79bd42c2a3ae7ae52d4e49a5 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:59:01.335676 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd16603-4a51-428d-9de0-c08bb67b6fbb admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:01.337431 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-8cd16603-4a51-428d-9de0-c08bb67b6fbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:01.371466 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd16603-4a51-428d-9de0-c08bb67b6fbb admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:01.592269 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aef10fb-c171-4d7d-8735-2d02a8b77e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:01.595353 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef10fb-c171-4d7d-8735-2d02a8b77e16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:01.595683 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef10fb-c171-4d7d-8735-2d02a8b77e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:01.596009 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef10fb-c171-4d7d-8735-2d02a8b77e16 admin admin] PATH_INFO: `/v3/projects/1cf85d202f4c41139c99e12310aec5d2` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:01.598033 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aef10fb-c171-4d7d-8735-2d02a8b77e16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cf85d202f4c41139c99e12310aec5d2)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:01.602640 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aef10fb-c171-4d7d-8735-2d02a8b77e16 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:01.860542 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-5aef10fb-c171-4d7d-8735-2d02a8b77e16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cf85d202f4c41139c99e12310aec5d2', 'request_id': 'req-5aef10fb-c171-4d7d-8735-2d02a8b77e16'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:01.872766 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-5aef10fb-c171-4d7d-8735-2d02a8b77e16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cf85d202f4c41139c99e12310aec5d2', 'request_id': 'req-5aef10fb-c171-4d7d-8735-2d02a8b77e16'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:01.957043 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-5aef10fb-c171-4d7d-8735-2d02a8b77e16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cf85d202f4c41139c99e12310aec5d2', 'request_id': 'req-5aef10fb-c171-4d7d-8735-2d02a8b77e16'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:01.975972 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-5aef10fb-c171-4d7d-8735-2d02a8b77e16 admin admin] The token cache is being invalidate because project 1cf85d202f4c41139c99e12310aec5d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:59:01.976406 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-5aef10fb-c171-4d7d-8735-2d02a8b77e16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:01.977421 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-5aef10fb-c171-4d7d-8735-2d02a8b77e16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:01.992976 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1153/2333] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:59:01 2022] DELETE /identity/v3/projects/1cf85d202f4c41139c99e12310aec5d2 => generated 0 bytes in 658 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:59:01.998542 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a19c5a-095c-4a37-98da-0cddd1df711a None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:01.998542 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a19c5a-095c-4a37-98da-0cddd1df711a None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:01.999256 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a19c5a-095c-4a37-98da-0cddd1df711a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:02.049937 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-62a19c5a-095c-4a37-98da-0cddd1df711a None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:59:02.054321 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-62a19c5a-095c-4a37-98da-0cddd1df711a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:02.182074 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1181/2334] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:59:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:59:02.189680 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a8f402-78d3-48d2-854b-8c2eec80c1c8 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:02.189955 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a8f402-78d3-48d2-854b-8c2eec80c1c8 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:02.190240 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a8f402-78d3-48d2-854b-8c2eec80c1c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:02.225624 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-e1a8f402-78d3-48d2-854b-8c2eec80c1c8 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:59:02.231238 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-e1a8f402-78d3-48d2-854b-8c2eec80c1c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:02.269149 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1154/2335] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:59:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:59:02.274286 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a19c5a-095c-4a37-98da-0cddd1df711a None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:02.288025 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a19c5a-095c-4a37-98da-0cddd1df711a None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:02.309806 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67558026-a404-4a1e-9634-ae3474887ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:02.310559 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67558026-a404-4a1e-9634-ae3474887ff2 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:02.310932 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67558026-a404-4a1e-9634-ae3474887ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:02.311639 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67558026-a404-4a1e-9634-ae3474887ff2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:02.312095 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67558026-a404-4a1e-9634-ae3474887ff2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:59:02.312443 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67558026-a404-4a1e-9634-ae3474887ff2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:02.324988 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67558026-a404-4a1e-9634-ae3474887ff2 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:02.345496 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a8f402-78d3-48d2-854b-8c2eec80c1c8 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:02.353714 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1182/2336] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 01:59:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:59:02.363200 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a8f402-78d3-48d2-854b-8c2eec80c1c8 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:02.378196 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbcb6ca-6cfc-435f-b25e-9666e0bdd1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:02.379311 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbcb6ca-6cfc-435f-b25e-9666e0bdd1d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:02.380474 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbcb6ca-6cfc-435f-b25e-9666e0bdd1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:02.380971 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbcb6ca-6cfc-435f-b25e-9666e0bdd1d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:02.382451 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbcb6ca-6cfc-435f-b25e-9666e0bdd1d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:02.387089 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbcb6ca-6cfc-435f-b25e-9666e0bdd1d1 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:02.413286 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:59:02.464464 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1155/2337] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:59:02 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:59:02.468437 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67558026-a404-4a1e-9634-ae3474887ff2 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:02.492295 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67558026-a404-4a1e-9634-ae3474887ff2 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:02.654909 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e23123-a8be-4c7f-9497-e3df119685f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:02.656201 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e23123-a8be-4c7f-9497-e3df119685f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:02.656575 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e23123-a8be-4c7f-9497-e3df119685f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:02.657011 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e23123-a8be-4c7f-9497-e3df119685f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:02.659301 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e23123-a8be-4c7f-9497-e3df119685f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:02.664085 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e23123-a8be-4c7f-9497-e3df119685f7 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:02.711292 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1183/2338] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:59:02 2022] POST /identity/v3/users => generated 287 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:59:02.712576 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbcb6ca-6cfc-435f-b25e-9666e0bdd1d1 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:02.733909 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbcb6ca-6cfc-435f-b25e-9666e0bdd1d1 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:02.756957 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addf0356-016e-433c-a403-e8bba9b9cffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:02.761232 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf0356-016e-433c-a403-e8bba9b9cffa admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:02.761592 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf0356-016e-433c-a403-e8bba9b9cffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:02.761883 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf0356-016e-433c-a403-e8bba9b9cffa admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:02.764730 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addf0356-016e-433c-a403-e8bba9b9cffa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:02.772142 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addf0356-016e-433c-a403-e8bba9b9cffa admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:02.815389 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1156/2339] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:59:02 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:59:02.823562 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e23123-a8be-4c7f-9497-e3df119685f7 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:02.839410 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e23123-a8be-4c7f-9497-e3df119685f7 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:02.855467 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7a796f-53d2-4505-a253-871e66d1cfb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:02.856169 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a796f-53d2-4505-a253-871e66d1cfb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:02.856448 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a796f-53d2-4505-a253-871e66d1cfb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:02.856718 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a796f-53d2-4505-a253-871e66d1cfb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:02.857456 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7a796f-53d2-4505-a253-871e66d1cfb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:02.859081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7a796f-53d2-4505-a253-871e66d1cfb8 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:02.879527 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1184/2340] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:59:02 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:59:02.884476 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addf0356-016e-433c-a403-e8bba9b9cffa admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:02.908473 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addf0356-016e-433c-a403-e8bba9b9cffa admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:02.920004 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f41c820-fa08-4d61-96d6-c32c47030e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:02.922183 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41c820-fa08-4d61-96d6-c32c47030e8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:02.922332 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41c820-fa08-4d61-96d6-c32c47030e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:02.923923 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41c820-fa08-4d61-96d6-c32c47030e8a admin admin] PATH_INFO: `/v3/projects/902d29e863af4f6dae1a2f75e6f02fa2/users/04b3f4250e7f491fba1d4533717cbaa9/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:02.958957 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f41c820-fa08-4d61-96d6-c32c47030e8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=902d29e863af4f6dae1a2f75e6f02fa2, user_id=04b3f4250e7f491fba1d4533717cbaa9, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:02.965212 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f41c820-fa08-4d61-96d6-c32c47030e8a admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:03.026920 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1157/2341] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:59:02 2022] PUT /identity/v3/projects/902d29e863af4f6dae1a2f75e6f02fa2/users/04b3f4250e7f491fba1d4533717cbaa9/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:59:03.038646 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7884d5-5c88-4ce9-b2f2-c34d6bc41826 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:03.039245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7884d5-5c88-4ce9-b2f2-c34d6bc41826 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:03.039245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7884d5-5c88-4ce9-b2f2-c34d6bc41826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:03.134391 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-8e7884d5-5c88-4ce9-b2f2-c34d6bc41826 None None] MFA Rules not processed for user `04b3f4250e7f491fba1d4533717cbaa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:59:03.142018 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-8e7884d5-5c88-4ce9-b2f2-c34d6bc41826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:03.168888 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:59:03.489859 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1185/2342] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:59:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 452 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:59:03.493792 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f41c820-fa08-4d61-96d6-c32c47030e8a admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:03.824931 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f41c820-fa08-4d61-96d6-c32c47030e8a admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:03.841901 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0d4d3a-b7cf-4aa1-a7cd-0b22fad1ad8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:03.842421 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d4d3a-b7cf-4aa1-a7cd-0b22fad1ad8c admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:03.842700 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d4d3a-b7cf-4aa1-a7cd-0b22fad1ad8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:03.842970 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d4d3a-b7cf-4aa1-a7cd-0b22fad1ad8c admin admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:03.845727 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0d4d3a-b7cf-4aa1-a7cd-0b22fad1ad8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:03.854434 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0d4d3a-b7cf-4aa1-a7cd-0b22fad1ad8c admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:03.933933 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1158/2343] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:59:03 2022] POST /identity/v3/projects => generated 373 bytes in 440 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:59:03.937160 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7884d5-5c88-4ce9-b2f2-c34d6bc41826 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:03.959944 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7884d5-5c88-4ce9-b2f2-c34d6bc41826 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:04.102013 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db2d77f-438d-4d1b-b5f5-6d947ddb3207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:04.103853 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2d77f-438d-4d1b-b5f5-6d947ddb3207 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:04.104391 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2d77f-438d-4d1b-b5f5-6d947ddb3207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:04.104989 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db2d77f-438d-4d1b-b5f5-6d947ddb3207 admin admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:04.106276 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db2d77f-438d-4d1b-b5f5-6d947ddb3207 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:04.109663 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db2d77f-438d-4d1b-b5f5-6d947ddb3207 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:04.146381 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1186/2344] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:59:03 2022] POST /identity/v3/users => generated 293 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 01:59:04.149585 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0d4d3a-b7cf-4aa1-a7cd-0b22fad1ad8c admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:04.165069 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0d4d3a-b7cf-4aa1-a7cd-0b22fad1ad8c admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:04.191664 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebac2f4-34e5-4ed9-95eb-75a4dba893c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:04.192535 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebac2f4-34e5-4ed9-95eb-75a4dba893c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:04.192825 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebac2f4-34e5-4ed9-95eb-75a4dba893c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:04.193280 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebac2f4-34e5-4ed9-95eb-75a4dba893c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:04.194179 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebac2f4-34e5-4ed9-95eb-75a4dba893c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:04.197560 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebac2f4-34e5-4ed9-95eb-75a4dba893c6 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:04.223273 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1159/2345] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:59:04 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:59:04.227071 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db2d77f-438d-4d1b-b5f5-6d947ddb3207 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:04.245776 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db2d77f-438d-4d1b-b5f5-6d947ddb3207 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:04.260224 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ade029-335d-41b0-b1d9-e37379543b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:04.264658 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ade029-335d-41b0-b1d9-e37379543b01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:04.266012 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ade029-335d-41b0-b1d9-e37379543b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:04.266598 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ade029-335d-41b0-b1d9-e37379543b01 admin admin] PATH_INFO: `/v3/projects/c002097dac05452098dde7502a967c6f/users/4dbca9f928454fef94a9921d13ad5176/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:04.308059 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ade029-335d-41b0-b1d9-e37379543b01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c002097dac05452098dde7502a967c6f, user_id=4dbca9f928454fef94a9921d13ad5176, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:04.310896 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ade029-335d-41b0-b1d9-e37379543b01 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:04.363553 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1187/2346] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:59:04 2022] PUT /identity/v3/projects/c002097dac05452098dde7502a967c6f/users/4dbca9f928454fef94a9921d13ad5176/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:59:04.368467 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebac2f4-34e5-4ed9-95eb-75a4dba893c6 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:04.385225 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebac2f4-34e5-4ed9-95eb-75a4dba893c6 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:04.508000 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:59:04.603179 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9612f663-02c2-4776-b61f-1b47828c5fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:04.603906 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612f663-02c2-4776-b61f-1b47828c5fa5 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:04.604406 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612f663-02c2-4776-b61f-1b47828c5fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:04.604685 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612f663-02c2-4776-b61f-1b47828c5fa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:04.605677 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9612f663-02c2-4776-b61f-1b47828c5fa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:04.607686 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9612f663-02c2-4776-b61f-1b47828c5fa5 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:04.639746 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1160/2347] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:59:04 2022] GET /identity/v3/roles => generated 1420 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:59:04.643176 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ade029-335d-41b0-b1d9-e37379543b01 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:04.669769 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ade029-335d-41b0-b1d9-e37379543b01 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:04.685510 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8598ddc-213b-460c-9ce6-833e75e9659a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:04.705088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8598ddc-213b-460c-9ce6-833e75e9659a admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:04.706650 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8598ddc-213b-460c-9ce6-833e75e9659a admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:04.707873 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8598ddc-213b-460c-9ce6-833e75e9659a admin admin] PATH_INFO: `/v3/projects/c002097dac05452098dde7502a967c6f/users/4dbca9f928454fef94a9921d13ad5176/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:04.722150 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8598ddc-213b-460c-9ce6-833e75e9659a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c002097dac05452098dde7502a967c6f, user_id=4dbca9f928454fef94a9921d13ad5176, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:04.733341 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8598ddc-213b-460c-9ce6-833e75e9659a admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:04.770649 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1188/2348] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:59:04 2022] PUT /identity/v3/projects/c002097dac05452098dde7502a967c6f/users/4dbca9f928454fef94a9921d13ad5176/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 01:59:04.781749 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bea3b4-9fc2-4fbd-abd9-ae182bf86a93 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:04.782577 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bea3b4-9fc2-4fbd-abd9-ae182bf86a93 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:04.783197 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bea3b4-9fc2-4fbd-abd9-ae182bf86a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:04.828274 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-70bea3b4-9fc2-4fbd-abd9-ae182bf86a93 None None] MFA Rules not processed for user `4dbca9f928454fef94a9921d13ad5176`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:59:04.830054 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-70bea3b4-9fc2-4fbd-abd9-ae182bf86a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:04.932291 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1161/2349] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:59:04 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:59:04.944317 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8598ddc-213b-460c-9ce6-833e75e9659a admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:04.948693 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-b8598ddc-213b-460c-9ce6-833e75e9659a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:05.054318 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:59:05.098940 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8598ddc-213b-460c-9ce6-833e75e9659a admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:05.104237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a00471-f021-4d10-afe2-f8a0c7497985 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:05.105733 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a00471-f021-4d10-afe2-f8a0c7497985 service cinder] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:05.107176 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a00471-f021-4d10-afe2-f8a0c7497985 service cinder] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:05.107176 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a00471-f021-4d10-afe2-f8a0c7497985 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:05.250814 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a00471-f021-4d10-afe2-f8a0c7497985 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:05.252660 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a00471-f021-4d10-afe2-f8a0c7497985 service cinder] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:05.264947 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1189/2350] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:59:04 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:59:05.590347 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bea3b4-9fc2-4fbd-abd9-ae182bf86a93 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:05.591902 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-70bea3b4-9fc2-4fbd-abd9-ae182bf86a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:05.732713 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bea3b4-9fc2-4fbd-abd9-ae182bf86a93 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:05.742447 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a44202-de01-4845-a822-a5e3bea63490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:05.743522 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a44202-de01-4845-a822-a5e3bea63490 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:05.743937 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a44202-de01-4845-a822-a5e3bea63490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:05.744218 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a44202-de01-4845-a822-a5e3bea63490 admin admin] PATH_INFO: `/v3/users/f3b443fd07d94499b6c9b8fa990c2d74` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:05.745638 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a44202-de01-4845-a822-a5e3bea63490 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3b443fd07d94499b6c9b8fa990c2d74)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:05.748857 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a44202-de01-4845-a822-a5e3bea63490 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:05.845125 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-e1a44202-de01-4845-a822-a5e3bea63490 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3b443fd07d94499b6c9b8fa990c2d74'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:05.862547 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:59:05.869585 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-e1a44202-de01-4845-a822-a5e3bea63490 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3b443fd07d94499b6c9b8fa990c2d74'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:05.884308 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-e1a44202-de01-4845-a822-a5e3bea63490 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3b443fd07d94499b6c9b8fa990c2d74'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:05.910189 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-e1a44202-de01-4845-a822-a5e3bea63490 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3b443fd07d94499b6c9b8fa990c2d74'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:05.911087 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-e1a44202-de01-4845-a822-a5e3bea63490 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3b443fd07d94499b6c9b8fa990c2d74'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:05.919923 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1162/2351] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:59:05 2022] DELETE /identity/v3/users/f3b443fd07d94499b6c9b8fa990c2d74 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:59:05.925910 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8757c4c-db2b-443d-aef6-6eb171bac690 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:05.926770 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8757c4c-db2b-443d-aef6-6eb171bac690 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:05.927138 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8757c4c-db2b-443d-aef6-6eb171bac690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:05.968814 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-d8757c4c-db2b-443d-aef6-6eb171bac690 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:59:05.970865 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-d8757c4c-db2b-443d-aef6-6eb171bac690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:06.107257 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1190/2352] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:59:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:59:06.120635 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a44202-de01-4845-a822-a5e3bea63490 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:06.123088 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-e1a44202-de01-4845-a822-a5e3bea63490 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:06.401674 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a44202-de01-4845-a822-a5e3bea63490 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:06.412080 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab1ddc7-e68a-49a9-96ab-8435e1778300 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:06.412777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1ddc7-e68a-49a9-96ab-8435e1778300 service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:06.413055 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1ddc7-e68a-49a9-96ab-8435e1778300 service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:06.413335 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1ddc7-e68a-49a9-96ab-8435e1778300 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:06.427800 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab1ddc7-e68a-49a9-96ab-8435e1778300 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:06.429940 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab1ddc7-e68a-49a9-96ab-8435e1778300 service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:06.454716 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1163/2353] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:59:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:59:07.360036 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8757c4c-db2b-443d-aef6-6eb171bac690 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:07.362465 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-d8757c4c-db2b-443d-aef6-6eb171bac690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:07.393124 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8757c4c-db2b-443d-aef6-6eb171bac690 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:07.398567 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84bcfb8-33ee-4031-b37d-36a9c18d290a None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:07.399015 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84bcfb8-33ee-4031-b37d-36a9c18d290a None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:07.399242 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84bcfb8-33ee-4031-b37d-36a9c18d290a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:07.415176 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f048e678-1cd2-4c8c-9cc9-7597a33a7059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:07.416450 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f048e678-1cd2-4c8c-9cc9-7597a33a7059 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:07.417064 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f048e678-1cd2-4c8c-9cc9-7597a33a7059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:07.417413 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f048e678-1cd2-4c8c-9cc9-7597a33a7059 admin admin] PATH_INFO: `/v3/projects/243b8841465e4e798163ba2f0bb1d33c` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:07.418727 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f048e678-1cd2-4c8c-9cc9-7597a33a7059 admin admin] RBAC: Authorizing `identity:delete_project(project_id=243b8841465e4e798163ba2f0bb1d33c)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:07.431115 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f048e678-1cd2-4c8c-9cc9-7597a33a7059 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:07.454726 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-d84bcfb8-33ee-4031-b37d-36a9c18d290a None None] MFA Rules not processed for user `c14a81624a264ab280d46d063ae676ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:59:07.458201 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-d84bcfb8-33ee-4031-b37d-36a9c18d290a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:07.566007 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-f048e678-1cd2-4c8c-9cc9-7597a33a7059 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '243b8841465e4e798163ba2f0bb1d33c', 'request_id': 'req-f048e678-1cd2-4c8c-9cc9-7597a33a7059'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:07.587534 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-f048e678-1cd2-4c8c-9cc9-7597a33a7059 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '243b8841465e4e798163ba2f0bb1d33c', 'request_id': 'req-f048e678-1cd2-4c8c-9cc9-7597a33a7059'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:07.621543 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-f048e678-1cd2-4c8c-9cc9-7597a33a7059 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '243b8841465e4e798163ba2f0bb1d33c', 'request_id': 'req-f048e678-1cd2-4c8c-9cc9-7597a33a7059'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:07.640133 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1164/2354] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:59:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:59:07.641777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84bcfb8-33ee-4031-b37d-36a9c18d290a None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:07.643436 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-d84bcfb8-33ee-4031-b37d-36a9c18d290a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:07.660603 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-f048e678-1cd2-4c8c-9cc9-7597a33a7059 admin admin] The token cache is being invalidate because project 243b8841465e4e798163ba2f0bb1d33c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:59:07.660904 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-f048e678-1cd2-4c8c-9cc9-7597a33a7059 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:07.662916 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-f048e678-1cd2-4c8c-9cc9-7597a33a7059 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:07.663980 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84bcfb8-33ee-4031-b37d-36a9c18d290a None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:07.668780 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1191/2355] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:59:07 2022] DELETE /identity/v3/projects/243b8841465e4e798163ba2f0bb1d33c => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:59:07.676851 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3239b16-ddf1-4197-8010-8475d742e831 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:07.677138 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3239b16-ddf1-4197-8010-8475d742e831 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:07.677412 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3239b16-ddf1-4197-8010-8475d742e831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:07.686383 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:59:07.736698 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:59:07.761186 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61983e81-2c84-47d0-bf81-3c91209945a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:07.762896 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61983e81-2c84-47d0-bf81-3c91209945a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:07.764221 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61983e81-2c84-47d0-bf81-3c91209945a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:07.764932 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61983e81-2c84-47d0-bf81-3c91209945a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:07.765987 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61983e81-2c84-47d0-bf81-3c91209945a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:07.768472 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61983e81-2c84-47d0-bf81-3c91209945a7 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:07.769108 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-d3239b16-ddf1-4197-8010-8475d742e831 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:59:07.775195 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-d3239b16-ddf1-4197-8010-8475d742e831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:07.805445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1192/2356] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:59:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:59:07.807328 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07411ac8-9ee4-4097-aee2-6a60b4789785 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:07.807615 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07411ac8-9ee4-4097-aee2-6a60b4789785 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:07.807884 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07411ac8-9ee4-4097-aee2-6a60b4789785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:07.831613 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1165/2357] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:59:07 2022] POST /identity/v3/projects => generated 379 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:59:07.834864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-07411ac8-9ee4-4097-aee2-6a60b4789785 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:59:07.836001 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61983e81-2c84-47d0-bf81-3c91209945a7 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:07.836266 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-07411ac8-9ee4-4097-aee2-6a60b4789785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:07.838773 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-61983e81-2c84-47d0-bf81-3c91209945a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:08.034234 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1193/2358] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:59:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:59:08.053545 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07411ac8-9ee4-4097-aee2-6a60b4789785 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:08.118104 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07411ac8-9ee4-4097-aee2-6a60b4789785 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:08.134847 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61983e81-2c84-47d0-bf81-3c91209945a7 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:08.140422 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc52a13d-2541-4262-9563-a640dd39f3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:08.141097 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc52a13d-2541-4262-9563-a640dd39f3e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:08.141468 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc52a13d-2541-4262-9563-a640dd39f3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:08.141722 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc52a13d-2541-4262-9563-a640dd39f3e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:08.143755 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc52a13d-2541-4262-9563-a640dd39f3e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:59:08.144065 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc52a13d-2541-4262-9563-a640dd39f3e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:08.146238 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc52a13d-2541-4262-9563-a640dd39f3e8 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:08.148723 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70b13df-6c76-4eff-9fba-fa87086c42e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:08.150059 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b13df-6c76-4eff-9fba-fa87086c42e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:08.156659 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b13df-6c76-4eff-9fba-fa87086c42e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:08.156999 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b13df-6c76-4eff-9fba-fa87086c42e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:08.157793 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70b13df-6c76-4eff-9fba-fa87086c42e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:08.160316 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70b13df-6c76-4eff-9fba-fa87086c42e0 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:08.167788 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1194/2359] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 01:59:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:59:08.172733 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc52a13d-2541-4262-9563-a640dd39f3e8 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:08.184801 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc52a13d-2541-4262-9563-a640dd39f3e8 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:08.190467 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849417a6-7d7b-4a21-896b-bfaf914c3c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:08.191516 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849417a6-7d7b-4a21-896b-bfaf914c3c6b admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:08.192726 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849417a6-7d7b-4a21-896b-bfaf914c3c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:08.192726 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849417a6-7d7b-4a21-896b-bfaf914c3c6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:08.192891 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849417a6-7d7b-4a21-896b-bfaf914c3c6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:08.207917 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849417a6-7d7b-4a21-896b-bfaf914c3c6b admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:08.218695 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1166/2360] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:59:07 2022] POST /identity/v3/users => generated 296 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 01:59:08.219747 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70b13df-6c76-4eff-9fba-fa87086c42e0 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:08.250096 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70b13df-6c76-4eff-9fba-fa87086c42e0 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:08.267472 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184cbff2-62c2-4466-af05-019295f5ac78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:08.268159 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184cbff2-62c2-4466-af05-019295f5ac78 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:08.268439 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184cbff2-62c2-4466-af05-019295f5ac78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:08.268715 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184cbff2-62c2-4466-af05-019295f5ac78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:08.269940 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184cbff2-62c2-4466-af05-019295f5ac78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:08.271196 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184cbff2-62c2-4466-af05-019295f5ac78 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:08.277495 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1195/2361] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:59:08 2022] POST /identity/v3/projects => generated 381 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:59:08.281897 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849417a6-7d7b-4a21-896b-bfaf914c3c6b admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:08.295841 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1167/2362] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:59:08 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:59:08.300409 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184cbff2-62c2-4466-af05-019295f5ac78 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:08.314939 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184cbff2-62c2-4466-af05-019295f5ac78 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:08.416517 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849417a6-7d7b-4a21-896b-bfaf914c3c6b admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:08.421161 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b713fba2-cb6d-4e64-9e81-e36193f22548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:08.424597 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713fba2-cb6d-4e64-9e81-e36193f22548 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:08.425124 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713fba2-cb6d-4e64-9e81-e36193f22548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:08.425586 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713fba2-cb6d-4e64-9e81-e36193f22548 admin admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:08.426518 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b713fba2-cb6d-4e64-9e81-e36193f22548 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:08.428798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b713fba2-cb6d-4e64-9e81-e36193f22548 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:08.445359 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b8d24-bad9-449f-8375-d14adb1e926e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:08.446337 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b8d24-bad9-449f-8375-d14adb1e926e admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:08.446627 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b8d24-bad9-449f-8375-d14adb1e926e admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:08.446956 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b8d24-bad9-449f-8375-d14adb1e926e admin admin] PATH_INFO: `/v3/projects/4a08609fee414568a1ead282fbddce87/users/c5a61d0f16044d15b3a26cfa30a7f2fc/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:08.482583 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4b8d24-bad9-449f-8375-d14adb1e926e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a08609fee414568a1ead282fbddce87, user_id=c5a61d0f16044d15b3a26cfa30a7f2fc, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:08.485513 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4b8d24-bad9-449f-8375-d14adb1e926e admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:08.511969 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1196/2363] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:59:08 2022] POST /identity/v3/users => generated 291 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 01:59:08.516294 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b713fba2-cb6d-4e64-9e81-e36193f22548 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:08.520148 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1168/2364] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:59:08 2022] PUT /identity/v3/projects/4a08609fee414568a1ead282fbddce87/users/c5a61d0f16044d15b3a26cfa30a7f2fc/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:59:08.523831 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b8d24-bad9-449f-8375-d14adb1e926e admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:08.539081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b713fba2-cb6d-4e64-9e81-e36193f22548 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:08.585743 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:59:08.601080 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:59:08.640626 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07476bcf-0522-44e8-bb1f-3a4c22037939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:08.642394 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07476bcf-0522-44e8-bb1f-3a4c22037939 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:08.643586 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07476bcf-0522-44e8-bb1f-3a4c22037939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:08.644433 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07476bcf-0522-44e8-bb1f-3a4c22037939 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:08.645920 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07476bcf-0522-44e8-bb1f-3a4c22037939 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:08.648386 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b8d24-bad9-449f-8375-d14adb1e926e admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:08.651026 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07476bcf-0522-44e8-bb1f-3a4c22037939 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:08.652788 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fea007-b17a-4309-a360-5ca8cf216c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:08.653676 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fea007-b17a-4309-a360-5ca8cf216c01 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:08.654525 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fea007-b17a-4309-a360-5ca8cf216c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:08.654525 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fea007-b17a-4309-a360-5ca8cf216c01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:08.655349 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fea007-b17a-4309-a360-5ca8cf216c01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:08.657268 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fea007-b17a-4309-a360-5ca8cf216c01 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:08.670006 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1169/2365] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:59:08 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:59:08.670629 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1197/2366] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:59:08 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:59:08.670806 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fea007-b17a-4309-a360-5ca8cf216c01 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:08.674830 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07476bcf-0522-44e8-bb1f-3a4c22037939 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:08.679648 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-55fea007-b17a-4309-a360-5ca8cf216c01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:08.687924 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07476bcf-0522-44e8-bb1f-3a4c22037939 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:08.712178 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fea007-b17a-4309-a360-5ca8cf216c01 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:08.714651 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c1dc58-7fce-4cc8-8e39-c17c08c8b6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:08.717706 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1dc58-7fce-4cc8-8e39-c17c08c8b6b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:08.718922 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1dc58-7fce-4cc8-8e39-c17c08c8b6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:08.719383 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1dc58-7fce-4cc8-8e39-c17c08c8b6b2 admin admin] PATH_INFO: `/v3/projects/4a08609fee414568a1ead282fbddce87/users/c5a61d0f16044d15b3a26cfa30a7f2fc/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:08.722181 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c1dc58-7fce-4cc8-8e39-c17c08c8b6b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a08609fee414568a1ead282fbddce87, user_id=c5a61d0f16044d15b3a26cfa30a7f2fc, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:08.725405 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c1dc58-7fce-4cc8-8e39-c17c08c8b6b2 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:08.726529 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d917558-43f4-4544-a305-a69343e567a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:08.730737 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d917558-43f4-4544-a305-a69343e567a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:08.733247 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d917558-43f4-4544-a305-a69343e567a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:08.733701 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d917558-43f4-4544-a305-a69343e567a9 admin admin] PATH_INFO: `/v3/users/04b3f4250e7f491fba1d4533717cbaa9` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:08.738921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d917558-43f4-4544-a305-a69343e567a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04b3f4250e7f491fba1d4533717cbaa9)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:08.749819 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d917558-43f4-4544-a305-a69343e567a9 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:08.762118 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1198/2367] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:59:08 2022] PUT /identity/v3/projects/4a08609fee414568a1ead282fbddce87/users/c5a61d0f16044d15b3a26cfa30a7f2fc/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:59:08.763444 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c1dc58-7fce-4cc8-8e39-c17c08c8b6b2 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:08.890954 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-8d917558-43f4-4544-a305-a69343e567a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04b3f4250e7f491fba1d4533717cbaa9'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:08.926081 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-8d917558-43f4-4544-a305-a69343e567a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04b3f4250e7f491fba1d4533717cbaa9'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:08.939465 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c1dc58-7fce-4cc8-8e39-c17c08c8b6b2 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:08.943687 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-8d917558-43f4-4544-a305-a69343e567a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04b3f4250e7f491fba1d4533717cbaa9'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:08.954705 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe92c5e-9630-4f79-be1b-3bb34485b24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:08.958087 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe92c5e-9630-4f79-be1b-3bb34485b24f admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:08.958372 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe92c5e-9630-4f79-be1b-3bb34485b24f admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:08.959783 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe92c5e-9630-4f79-be1b-3bb34485b24f admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:08.960984 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-8d917558-43f4-4544-a305-a69343e567a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04b3f4250e7f491fba1d4533717cbaa9'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:08.961994 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe92c5e-9630-4f79-be1b-3bb34485b24f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:08.962541 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-8d917558-43f4-4544-a305-a69343e567a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04b3f4250e7f491fba1d4533717cbaa9'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:08.965345 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe92c5e-9630-4f79-be1b-3bb34485b24f admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:08.981696 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1170/2368] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:59:08 2022] DELETE /identity/v3/users/04b3f4250e7f491fba1d4533717cbaa9 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:59:08.987752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96cdec7-dad2-45b7-923b-168fc240af11 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:08.988185 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96cdec7-dad2-45b7-923b-168fc240af11 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:08.988595 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96cdec7-dad2-45b7-923b-168fc240af11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:08.996138 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1199/2369] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:59:08 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:59:09.000593 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe92c5e-9630-4f79-be1b-3bb34485b24f admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:09.005513 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-1fe92c5e-9630-4f79-be1b-3bb34485b24f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:09.033882 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-d96cdec7-dad2-45b7-923b-168fc240af11 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:59:09.036697 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-d96cdec7-dad2-45b7-923b-168fc240af11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:09.037250 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe92c5e-9630-4f79-be1b-3bb34485b24f admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:09.242979 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d15e33-fce6-4a2e-acf9-12d1d50364b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:09.244175 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d15e33-fce6-4a2e-acf9-12d1d50364b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:09.244460 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d15e33-fce6-4a2e-acf9-12d1d50364b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:09.244733 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d15e33-fce6-4a2e-acf9-12d1d50364b8 admin admin] PATH_INFO: `/v3/projects/9b84d8e675cc4386bb229f124b399e03/users/b2138120a6814dc4935a03203d35cb6d/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:09.321459 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d15e33-fce6-4a2e-acf9-12d1d50364b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b84d8e675cc4386bb229f124b399e03, user_id=b2138120a6814dc4935a03203d35cb6d, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:09.324232 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d15e33-fce6-4a2e-acf9-12d1d50364b8 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:09.342635 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1171/2370] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:59:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 01:59:09.380086 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96cdec7-dad2-45b7-923b-168fc240af11 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:09.383254 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-d96cdec7-dad2-45b7-923b-168fc240af11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:09.397729 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1200/2371] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:59:08 2022] PUT /identity/v3/projects/9b84d8e675cc4386bb229f124b399e03/users/b2138120a6814dc4935a03203d35cb6d/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:59:09.409164 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc57aa4-a478-4e4d-aedd-1e8387fd1956 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:09.409164 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc57aa4-a478-4e4d-aedd-1e8387fd1956 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:09.409164 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc57aa4-a478-4e4d-aedd-1e8387fd1956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:09.469865 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-bfc57aa4-a478-4e4d-aedd-1e8387fd1956 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:59:09.477735 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-bfc57aa4-a478-4e4d-aedd-1e8387fd1956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:09.506428 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:59:09.539469 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96cdec7-dad2-45b7-923b-168fc240af11 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:09.594322 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:59:09.612333 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b27ee8-25e9-44da-8024-291d12e0483b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:09.613329 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b27ee8-25e9-44da-8024-291d12e0483b service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:09.613758 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b27ee8-25e9-44da-8024-291d12e0483b service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:09.614192 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b27ee8-25e9-44da-8024-291d12e0483b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:09.658038 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1201/2372] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:59:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:59:09.686211 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc57aa4-a478-4e4d-aedd-1e8387fd1956 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:09.688568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-bfc57aa4-a478-4e4d-aedd-1e8387fd1956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:09.693393 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b27ee8-25e9-44da-8024-291d12e0483b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:09.695815 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b27ee8-25e9-44da-8024-291d12e0483b service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:09.710640 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1172/2373] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:59:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 01:59:09.715937 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc57aa4-a478-4e4d-aedd-1e8387fd1956 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:09.728048 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd9ad2b-8589-4654-81a0-0833c9d4d571 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:09.729913 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd9ad2b-8589-4654-81a0-0833c9d4d571 service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:09.735505 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd9ad2b-8589-4654-81a0-0833c9d4d571 service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:09.736221 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd9ad2b-8589-4654-81a0-0833c9d4d571 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:09.761279 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd9ad2b-8589-4654-81a0-0833c9d4d571 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:09.762942 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd9ad2b-8589-4654-81a0-0833c9d4d571 service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:09.792463 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1202/2374] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:59:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 01:59:10.439069 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b27ee8-25e9-44da-8024-291d12e0483b service neutron] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:10.440596 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-48b27ee8-25e9-44da-8024-291d12e0483b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:10.467179 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b27ee8-25e9-44da-8024-291d12e0483b service neutron] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:10.474245 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aace1ac-ebcf-40fa-a53c-f39f054fdd7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:10.475139 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aace1ac-ebcf-40fa-a53c-f39f054fdd7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:10.475597 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aace1ac-ebcf-40fa-a53c-f39f054fdd7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:10.476024 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aace1ac-ebcf-40fa-a53c-f39f054fdd7c admin admin] PATH_INFO: `/v3/projects/902d29e863af4f6dae1a2f75e6f02fa2` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:10.480541 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aace1ac-ebcf-40fa-a53c-f39f054fdd7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=902d29e863af4f6dae1a2f75e6f02fa2)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:10.482607 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aace1ac-ebcf-40fa-a53c-f39f054fdd7c admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:10.615950 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-0aace1ac-ebcf-40fa-a53c-f39f054fdd7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '902d29e863af4f6dae1a2f75e6f02fa2', 'request_id': 'req-0aace1ac-ebcf-40fa-a53c-f39f054fdd7c'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:10.634672 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-0aace1ac-ebcf-40fa-a53c-f39f054fdd7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '902d29e863af4f6dae1a2f75e6f02fa2', 'request_id': 'req-0aace1ac-ebcf-40fa-a53c-f39f054fdd7c'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:10.647527 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-0aace1ac-ebcf-40fa-a53c-f39f054fdd7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '902d29e863af4f6dae1a2f75e6f02fa2', 'request_id': 'req-0aace1ac-ebcf-40fa-a53c-f39f054fdd7c'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:10.663422 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-0aace1ac-ebcf-40fa-a53c-f39f054fdd7c admin admin] The token cache is being invalidate because project 902d29e863af4f6dae1a2f75e6f02fa2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:59:10.663719 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-0aace1ac-ebcf-40fa-a53c-f39f054fdd7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:10.667019 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-0aace1ac-ebcf-40fa-a53c-f39f054fdd7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:10.670384 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1173/2375] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:59:10 2022] DELETE /identity/v3/projects/902d29e863af4f6dae1a2f75e6f02fa2 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:59:10.675030 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd9ad2b-8589-4654-81a0-0833c9d4d571 service neutron] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:10.704937 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd9ad2b-8589-4654-81a0-0833c9d4d571 service neutron] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:10.825478 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd7b76a-7e07-496b-8982-5332e09ba1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:10.826386 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd7b76a-7e07-496b-8982-5332e09ba1e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:10.826666 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd7b76a-7e07-496b-8982-5332e09ba1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:10.826937 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd7b76a-7e07-496b-8982-5332e09ba1e0 admin admin] PATH_INFO: `/v3/users/4dbca9f928454fef94a9921d13ad5176` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:10.829363 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd7b76a-7e07-496b-8982-5332e09ba1e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dbca9f928454fef94a9921d13ad5176)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:10.831353 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd7b76a-7e07-496b-8982-5332e09ba1e0 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:10.969273 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-abd7b76a-7e07-496b-8982-5332e09ba1e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dbca9f928454fef94a9921d13ad5176'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:10.995889 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-abd7b76a-7e07-496b-8982-5332e09ba1e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dbca9f928454fef94a9921d13ad5176'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:11.015350 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-abd7b76a-7e07-496b-8982-5332e09ba1e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dbca9f928454fef94a9921d13ad5176'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:11.029525 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-abd7b76a-7e07-496b-8982-5332e09ba1e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dbca9f928454fef94a9921d13ad5176'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:11.030874 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-abd7b76a-7e07-496b-8982-5332e09ba1e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dbca9f928454fef94a9921d13ad5176'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:11.049053 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1203/2376] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:59:10 2022] DELETE /identity/v3/users/4dbca9f928454fef94a9921d13ad5176 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:59:11.963138 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aace1ac-ebcf-40fa-a53c-f39f054fdd7c admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:11.965824 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-0aace1ac-ebcf-40fa-a53c-f39f054fdd7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:11.987579 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aace1ac-ebcf-40fa-a53c-f39f054fdd7c admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:12.076316 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:59:12.140619 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc04f3-3504-447d-b506-f43203d77f15 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:12.141641 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc04f3-3504-447d-b506-f43203d77f15 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:12.142741 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc04f3-3504-447d-b506-f43203d77f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:12.142926 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3a815-2d3a-4574-838f-0eee4b0b7ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:12.145229 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3a815-2d3a-4574-838f-0eee4b0b7ed6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:12.145509 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3a815-2d3a-4574-838f-0eee4b0b7ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:12.145784 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3a815-2d3a-4574-838f-0eee4b0b7ed6 admin admin] PATH_INFO: `/v3/projects/c002097dac05452098dde7502a967c6f` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:12.148784 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed3a815-2d3a-4574-838f-0eee4b0b7ed6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c002097dac05452098dde7502a967c6f)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:12.155348 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed3a815-2d3a-4574-838f-0eee4b0b7ed6 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:12.181084 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-59cc04f3-3504-447d-b506-f43203d77f15 None None] MFA Rules not processed for user `b2138120a6814dc4935a03203d35cb6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:59:12.183031 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-59cc04f3-3504-447d-b506-f43203d77f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:12.233882 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:59:12.276512 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-5ed3a815-2d3a-4574-838f-0eee4b0b7ed6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c002097dac05452098dde7502a967c6f', 'request_id': 'req-5ed3a815-2d3a-4574-838f-0eee4b0b7ed6'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:12.293459 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-5ed3a815-2d3a-4574-838f-0eee4b0b7ed6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c002097dac05452098dde7502a967c6f', 'request_id': 'req-5ed3a815-2d3a-4574-838f-0eee4b0b7ed6'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:12.308336 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1204/2377] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:59:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:59:12.312498 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-5ed3a815-2d3a-4574-838f-0eee4b0b7ed6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c002097dac05452098dde7502a967c6f', 'request_id': 'req-5ed3a815-2d3a-4574-838f-0eee4b0b7ed6'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:12.316469 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cc04f3-3504-447d-b506-f43203d77f15 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:12.321614 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-59cc04f3-3504-447d-b506-f43203d77f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:12.333869 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-5ed3a815-2d3a-4574-838f-0eee4b0b7ed6 admin admin] The token cache is being invalidate because project c002097dac05452098dde7502a967c6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:59:12.334605 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-5ed3a815-2d3a-4574-838f-0eee4b0b7ed6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:12.339920 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-5ed3a815-2d3a-4574-838f-0eee4b0b7ed6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:12.345671 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1174/2378] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:59:11 2022] DELETE /identity/v3/projects/c002097dac05452098dde7502a967c6f => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 01:59:12.347075 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78759e0e-1782-4130-b5b0-225f1e50ab91 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:12.347391 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78759e0e-1782-4130-b5b0-225f1e50ab91 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:12.347672 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78759e0e-1782-4130-b5b0-225f1e50ab91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:12.407464 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-78759e0e-1782-4130-b5b0-225f1e50ab91 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:59:12.410318 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-78759e0e-1782-4130-b5b0-225f1e50ab91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:12.413065 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cc04f3-3504-447d-b506-f43203d77f15 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:12.532039 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dd0c6f-6ad1-445a-be22-806fc7eeb745 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:12.532611 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd0c6f-6ad1-445a-be22-806fc7eeb745 service cinder] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:12.532893 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd0c6f-6ad1-445a-be22-806fc7eeb745 service cinder] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:12.533168 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd0c6f-6ad1-445a-be22-806fc7eeb745 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:12.540263 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-f0dd0c6f-6ad1-445a-be22-806fc7eeb745 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:12.570795 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1175/2379] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:59:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:59:12.580513 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd90fe2-1a4a-4607-a167-d881c1c1dced None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:12.583226 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd90fe2-1a4a-4607-a167-d881c1c1dced None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:12.583528 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd90fe2-1a4a-4607-a167-d881c1c1dced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:12.622730 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-dcd90fe2-1a4a-4607-a167-d881c1c1dced None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:59:12.624487 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-dcd90fe2-1a4a-4607-a167-d881c1c1dced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:12.676889 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1176/2380] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:59:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:59:12.685525 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd90fe2-1a4a-4607-a167-d881c1c1dced None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:12.697809 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dd0c6f-6ad1-445a-be22-806fc7eeb745 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:12.701596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dd0c6f-6ad1-445a-be22-806fc7eeb745 service cinder] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:12.706166 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd90fe2-1a4a-4607-a167-d881c1c1dced None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:12.713745 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddae3a0-7653-42c5-a9d5-1ee165b46845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:12.715097 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddae3a0-7653-42c5-a9d5-1ee165b46845 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:12.715765 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddae3a0-7653-42c5-a9d5-1ee165b46845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:12.716190 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddae3a0-7653-42c5-a9d5-1ee165b46845 admin admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:12.717057 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddae3a0-7653-42c5-a9d5-1ee165b46845 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 01:59:12.717566 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddae3a0-7653-42c5-a9d5-1ee165b46845 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:12.727907 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddae3a0-7653-42c5-a9d5-1ee165b46845 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:12.744860 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1205/2381] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:59:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:59:12.784894 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dd0c6f-6ad1-445a-be22-806fc7eeb745 service cinder] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:12.785199 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1177/2382] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 01:59:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 01:59:12.809229 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dd0c6f-6ad1-445a-be22-806fc7eeb745 service cinder] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:12.843950 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccf19a1-6796-4846-b744-fa017965595b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:12.844551 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf19a1-6796-4846-b744-fa017965595b admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:12.844836 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf19a1-6796-4846-b744-fa017965595b admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:12.845112 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf19a1-6796-4846-b744-fa017965595b admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:12.845895 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccf19a1-6796-4846-b744-fa017965595b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:12.847882 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccf19a1-6796-4846-b744-fa017965595b admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:12.923140 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1206/2383] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 01:59:12 2022] POST /identity/v3/projects => generated 399 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:59:12.927151 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddae3a0-7653-42c5-a9d5-1ee165b46845 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:12.941862 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddae3a0-7653-42c5-a9d5-1ee165b46845 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:13.023700 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:59:13.051197 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801ef8fb-2070-4240-b5e7-269b3a9010c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:13.052533 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ef8fb-2070-4240-b5e7-269b3a9010c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:13.052533 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ef8fb-2070-4240-b5e7-269b3a9010c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:13.052785 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ef8fb-2070-4240-b5e7-269b3a9010c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:13.053578 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801ef8fb-2070-4240-b5e7-269b3a9010c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:13.058653 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801ef8fb-2070-4240-b5e7-269b3a9010c5 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:13.093516 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1178/2384] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 01:59:12 2022] POST /identity/v3/users => generated 300 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 01:59:13.094791 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccf19a1-6796-4846-b744-fa017965595b admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:13.112695 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccf19a1-6796-4846-b744-fa017965595b admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:13.120726 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f46c62f-c15b-48eb-9604-6c7a8beef4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:13.121408 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46c62f-c15b-48eb-9604-6c7a8beef4c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:13.121691 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46c62f-c15b-48eb-9604-6c7a8beef4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:13.121967 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46c62f-c15b-48eb-9604-6c7a8beef4c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:13.122722 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f46c62f-c15b-48eb-9604-6c7a8beef4c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:13.128043 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f46c62f-c15b-48eb-9604-6c7a8beef4c6 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:13.152632 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801ef8fb-2070-4240-b5e7-269b3a9010c5 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:13.153056 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1207/2385] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:59:13 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:59:13.173377 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801ef8fb-2070-4240-b5e7-269b3a9010c5 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:13.187379 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ba7cda-d06e-40b8-b453-15765f5592f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:13.191363 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba7cda-d06e-40b8-b453-15765f5592f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:13.191363 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba7cda-d06e-40b8-b453-15765f5592f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:13.191363 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba7cda-d06e-40b8-b453-15765f5592f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:13.191363 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ba7cda-d06e-40b8-b453-15765f5592f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:13.191975 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ba7cda-d06e-40b8-b453-15765f5592f7 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:13.205002 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1179/2386] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 01:59:13 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 01:59:13.209288 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f46c62f-c15b-48eb-9604-6c7a8beef4c6 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:13.222117 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f46c62f-c15b-48eb-9604-6c7a8beef4c6 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:13.228211 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4784ab-9b0f-4ecc-a6c4-416ac59ee333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:13.229651 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4784ab-9b0f-4ecc-a6c4-416ac59ee333 admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:13.230330 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4784ab-9b0f-4ecc-a6c4-416ac59ee333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:13.230793 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4784ab-9b0f-4ecc-a6c4-416ac59ee333 admin admin] PATH_INFO: `/v3/projects/e08075248f6c4a5f8994843fa70030fd/users/049a64280a134d6da6062a33f18348ea/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:13.243816 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:59:13.254037 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4784ab-9b0f-4ecc-a6c4-416ac59ee333 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e08075248f6c4a5f8994843fa70030fd, user_id=049a64280a134d6da6062a33f18348ea, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:13.257870 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4784ab-9b0f-4ecc-a6c4-416ac59ee333 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:13.276994 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1208/2387] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 01:59:13 2022] PUT /identity/v3/projects/e08075248f6c4a5f8994843fa70030fd/users/049a64280a134d6da6062a33f18348ea/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:59:13.287641 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ea7b9d-c820-4383-b27f-1ae5a34b302b None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:13.288370 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ea7b9d-c820-4383-b27f-1ae5a34b302b None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:13.289059 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ea7b9d-c820-4383-b27f-1ae5a34b302b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:13.339726 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-21ea7b9d-c820-4383-b27f-1ae5a34b302b None None] MFA Rules not processed for user `049a64280a134d6da6062a33f18348ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:59:13.341007 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-21ea7b9d-c820-4383-b27f-1ae5a34b302b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:13.462801 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1180/2388] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:59:13 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:59:13.478398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4784ab-9b0f-4ecc-a6c4-416ac59ee333 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:13.480264 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-8b4784ab-9b0f-4ecc-a6c4-416ac59ee333 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:13.616800 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4784ab-9b0f-4ecc-a6c4-416ac59ee333 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:13.626444 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676569f2-44fc-4bad-96d9-683bbf9a2457 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:13.627589 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676569f2-44fc-4bad-96d9-683bbf9a2457 service cinder] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:13.627878 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676569f2-44fc-4bad-96d9-683bbf9a2457 service cinder] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:13.628158 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676569f2-44fc-4bad-96d9-683bbf9a2457 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:13.645478 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676569f2-44fc-4bad-96d9-683bbf9a2457 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:13.647573 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676569f2-44fc-4bad-96d9-683bbf9a2457 service cinder] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:13.663072 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1209/2389] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:59:13 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:59:14.584022 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50016d-bfbd-432e-9b8b-1d0aae852d67 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:14.585050 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50016d-bfbd-432e-9b8b-1d0aae852d67 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:14.585050 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50016d-bfbd-432e-9b8b-1d0aae852d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:14.665007 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-aa50016d-bfbd-432e-9b8b-1d0aae852d67 None None] MFA Rules not processed for user `b2138120a6814dc4935a03203d35cb6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:59:14.666568 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-aa50016d-bfbd-432e-9b8b-1d0aae852d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:14.949594 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1181/2390] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:59:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:59:15.067701 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676569f2-44fc-4bad-96d9-683bbf9a2457 service cinder] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:15.069189 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-676569f2-44fc-4bad-96d9-683bbf9a2457 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:15.244268 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676569f2-44fc-4bad-96d9-683bbf9a2457 service cinder] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:15.255688 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f83bb4-ef46-4e6b-a6c0-e0caffb2c72b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:15.256495 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f83bb4-ef46-4e6b-a6c0-e0caffb2c72b service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:15.256885 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f83bb4-ef46-4e6b-a6c0-e0caffb2c72b service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:15.257424 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f83bb4-ef46-4e6b-a6c0-e0caffb2c72b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:15.260210 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-b0f83bb4-ef46-4e6b-a6c0-e0caffb2c72b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:15.405272 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f83bb4-ef46-4e6b-a6c0-e0caffb2c72b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:15.407140 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f83bb4-ef46-4e6b-a6c0-e0caffb2c72b service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:15.444469 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1210/2391] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:59:15 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:59:19.057893 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220244e-26cd-4b4f-8bef-dbaa974a0b95 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:19.058491 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220244e-26cd-4b4f-8bef-dbaa974a0b95 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:19.058783 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220244e-26cd-4b4f-8bef-dbaa974a0b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:19.115035 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-6220244e-26cd-4b4f-8bef-dbaa974a0b95 None None] MFA Rules not processed for user `c5a61d0f16044d15b3a26cfa30a7f2fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:59:19.116581 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-6220244e-26cd-4b4f-8bef-dbaa974a0b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:19.295836 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:59:19.358837 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1182/2392] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:59:19 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:59:19.374179 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f83bb4-ef46-4e6b-a6c0-e0caffb2c72b service neutron] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:19.376043 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-b0f83bb4-ef46-4e6b-a6c0-e0caffb2c72b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:19.451251 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:59:19.513014 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f83bb4-ef46-4e6b-a6c0-e0caffb2c72b service neutron] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:19.522658 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641158a1-574c-46ec-bf57-94d4d9f6da24 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4e83df33b64a3ebf28f54e2d94fe63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:19.523677 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641158a1-574c-46ec-bf57-94d4d9f6da24 service nova] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:19.524143 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641158a1-574c-46ec-bf57-94d4d9f6da24 service nova] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:19.526171 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641158a1-574c-46ec-bf57-94d4d9f6da24 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:19.528732 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-641158a1-574c-46ec-bf57-94d4d9f6da24 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:19.612116 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641158a1-574c-46ec-bf57-94d4d9f6da24 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:19.614019 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641158a1-574c-46ec-bf57-94d4d9f6da24 service nova] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:19.625310 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1211/2393] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:59:19 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:59:21.496913 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6220244e-26cd-4b4f-8bef-dbaa974a0b95 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:21.587379 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6220244e-26cd-4b4f-8bef-dbaa974a0b95 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:21.592437 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e31e1-61cc-42ce-88af-106597a7e277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:21.593993 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e31e1-61cc-42ce-88af-106597a7e277 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:21.594506 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e31e1-61cc-42ce-88af-106597a7e277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:21.594933 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e31e1-61cc-42ce-88af-106597a7e277 admin admin] PATH_INFO: `/v3/users/049a64280a134d6da6062a33f18348ea` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:21.597153 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52e31e1-61cc-42ce-88af-106597a7e277 admin admin] RBAC: Authorizing `identity:delete_user(user_id=049a64280a134d6da6062a33f18348ea)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:21.599591 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52e31e1-61cc-42ce-88af-106597a7e277 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:21.698637 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-f52e31e1-61cc-42ce-88af-106597a7e277 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '049a64280a134d6da6062a33f18348ea'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:21.728571 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-f52e31e1-61cc-42ce-88af-106597a7e277 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '049a64280a134d6da6062a33f18348ea'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:21.746818 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-f52e31e1-61cc-42ce-88af-106597a7e277 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '049a64280a134d6da6062a33f18348ea'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:21.763528 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-f52e31e1-61cc-42ce-88af-106597a7e277 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '049a64280a134d6da6062a33f18348ea'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:21.764634 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-f52e31e1-61cc-42ce-88af-106597a7e277 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '049a64280a134d6da6062a33f18348ea'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:21.776469 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1183/2394] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 01:59:21 2022] DELETE /identity/v3/users/049a64280a134d6da6062a33f18348ea => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:59:21.783606 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03a173-04b1-4634-b0cb-81832cd0afec None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:21.784441 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03a173-04b1-4634-b0cb-81832cd0afec None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:21.785017 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03a173-04b1-4634-b0cb-81832cd0afec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:21.829346 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-ff03a173-04b1-4634-b0cb-81832cd0afec None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:59:21.836406 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-ff03a173-04b1-4634-b0cb-81832cd0afec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:21.982623 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1212/2395] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:59:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:59:22.085640 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e31e1-61cc-42ce-88af-106597a7e277 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:22.087954 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-f52e31e1-61cc-42ce-88af-106597a7e277 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:22.193317 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e31e1-61cc-42ce-88af-106597a7e277 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:22.198714 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caa4166-299d-4976-a076-f2ac24e33ae5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:22.200173 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa4166-299d-4976-a076-f2ac24e33ae5 service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:22.200173 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa4166-299d-4976-a076-f2ac24e33ae5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:22.200418 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa4166-299d-4976-a076-f2ac24e33ae5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:22.230624 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caa4166-299d-4976-a076-f2ac24e33ae5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:22.230624 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caa4166-299d-4976-a076-f2ac24e33ae5 service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:22.245236 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1184/2396] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:59:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:59:24.202268 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff03a173-04b1-4634-b0cb-81832cd0afec None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:24.203834 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-ff03a173-04b1-4634-b0cb-81832cd0afec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:24.230076 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff03a173-04b1-4634-b0cb-81832cd0afec None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:24.246014 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb7d276-c255-4ace-b3a0-5741302d0c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:24.246936 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb7d276-c255-4ace-b3a0-5741302d0c10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:24.247244 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb7d276-c255-4ace-b3a0-5741302d0c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:24.247695 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb7d276-c255-4ace-b3a0-5741302d0c10 admin admin] PATH_INFO: `/v3/projects/e08075248f6c4a5f8994843fa70030fd` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:24.249106 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb7d276-c255-4ace-b3a0-5741302d0c10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e08075248f6c4a5f8994843fa70030fd)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:24.251581 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb7d276-c255-4ace-b3a0-5741302d0c10 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:24.329926 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-fbb7d276-c255-4ace-b3a0-5741302d0c10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e08075248f6c4a5f8994843fa70030fd', 'request_id': 'req-fbb7d276-c255-4ace-b3a0-5741302d0c10'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:24.337566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-fbb7d276-c255-4ace-b3a0-5741302d0c10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e08075248f6c4a5f8994843fa70030fd', 'request_id': 'req-fbb7d276-c255-4ace-b3a0-5741302d0c10'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:24.348413 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-fbb7d276-c255-4ace-b3a0-5741302d0c10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e08075248f6c4a5f8994843fa70030fd', 'request_id': 'req-fbb7d276-c255-4ace-b3a0-5741302d0c10'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:24.357452 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:59:24.362656 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-fbb7d276-c255-4ace-b3a0-5741302d0c10 admin admin] The token cache is being invalidate because project e08075248f6c4a5f8994843fa70030fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 01:59:24.363017 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-fbb7d276-c255-4ace-b3a0-5741302d0c10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:24.364113 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-fbb7d276-c255-4ace-b3a0-5741302d0c10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 01:59:24.366403 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1213/2397] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 01:59:24 2022] DELETE /identity/v3/projects/e08075248f6c4a5f8994843fa70030fd => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 01:59:26.909068 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c19d6-c4d7-42b9-9217-354323b181e2 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:26.909619 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c19d6-c4d7-42b9-9217-354323b181e2 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:26.909850 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c19d6-c4d7-42b9-9217-354323b181e2 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:26.911844 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1185/2398] 10.222.0.26 () {54 vars in 921 bytes} [Thu Jun 23 01:59:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:59:26.918291 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e705905c-a9c9-4b67-92aa-3fd966c4c647 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:26.918829 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e705905c-a9c9-4b67-92aa-3fd966c4c647 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:26.918829 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e705905c-a9c9-4b67-92aa-3fd966c4c647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:27.010598 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-e705905c-a9c9-4b67-92aa-3fd966c4c647 None None] MFA Rules not processed for user `ae4e83df33b64a3ebf28f54e2d94fe63`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:59:27.011883 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-e705905c-a9c9-4b67-92aa-3fd966c4c647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:27.113996 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1214/2399] 10.222.0.26 () {58 vars in 1018 bytes} [Thu Jun 23 01:59:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:59:27.125642 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87c19d6-c4d7-42b9-9217-354323b181e2 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:27.127540 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-c87c19d6-c4d7-42b9-9217-354323b181e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:27.134951 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:59:27.190502 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87c19d6-c4d7-42b9-9217-354323b181e2 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:27.194326 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60ca70-69e6-4a38-969b-90fcbd9b909c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c081b4cf34146ca8eb197b62a133433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:27.194989 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60ca70-69e6-4a38-969b-90fcbd9b909c service glance] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:27.195348 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60ca70-69e6-4a38-969b-90fcbd9b909c service glance] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:27.195623 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60ca70-69e6-4a38-969b-90fcbd9b909c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:27.206219 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be60ca70-69e6-4a38-969b-90fcbd9b909c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:27.207684 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be60ca70-69e6-4a38-969b-90fcbd9b909c service glance] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:27.218212 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1186/2400] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:59:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 01:59:27.295292 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0b86e-bf1e-4284-bbc1-a3fa9179ef32 None None] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:27.295712 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0b86e-bf1e-4284-bbc1-a3fa9179ef32 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:27.295993 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0b86e-bf1e-4284-bbc1-a3fa9179ef32 None None] PATH_INFO: `/` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:27.297862 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1215/2401] 10.222.0.26 () {54 vars in 921 bytes} [Thu Jun 23 01:59:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:59:27.302622 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3a84b1-a346-40e1-8a39-079e35c7b714 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:27.302923 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3a84b1-a346-40e1-8a39-079e35c7b714 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:27.303198 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3a84b1-a346-40e1-8a39-079e35c7b714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:27.322456 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-6b3a84b1-a346-40e1-8a39-079e35c7b714 None None] MFA Rules not processed for user `948e4905de44473bb7a45e1b521cf095`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:59:27.323701 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-6b3a84b1-a346-40e1-8a39-079e35c7b714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:27.378362 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1187/2402] 10.222.0.26 () {58 vars in 1018 bytes} [Thu Jun 23 01:59:27 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 01:59:27.390504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee0b86e-bf1e-4284-bbc1-a3fa9179ef32 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:27.392452 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-fee0b86e-bf1e-4284-bbc1-a3fa9179ef32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:27.406633 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee0b86e-bf1e-4284-bbc1-a3fa9179ef32 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:27.410940 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ade62c6-5d7f-4345-8dd5-6709bc86b6fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:27.411600 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade62c6-5d7f-4345-8dd5-6709bc86b6fc service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:27.411877 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade62c6-5d7f-4345-8dd5-6709bc86b6fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:27.412143 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade62c6-5d7f-4345-8dd5-6709bc86b6fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:27.419756 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ade62c6-5d7f-4345-8dd5-6709bc86b6fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:27.421186 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ade62c6-5d7f-4345-8dd5-6709bc86b6fc service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:27.429276 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1216/2403] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:59:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:59:27.518708 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b7023-560a-45c4-b276-a35b1c34c814 None None] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:27.518971 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b7023-560a-45c4-b276-a35b1c34c814 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:27.519321 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b7023-560a-45c4-b276-a35b1c34c814 None None] PATH_INFO: `/` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:27.520702 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1188/2404] 10.222.0.26 () {54 vars in 921 bytes} [Thu Jun 23 01:59:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 01:59:27.526929 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c22d55-9478-46d2-8521-48a5aad5863b None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:27.527342 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c22d55-9478-46d2-8521-48a5aad5863b None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:27.527622 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c22d55-9478-46d2-8521-48a5aad5863b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:27.547702 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-a0c22d55-9478-46d2-8521-48a5aad5863b None None] MFA Rules not processed for user `ae4e83df33b64a3ebf28f54e2d94fe63`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:59:27.548910 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-a0c22d55-9478-46d2-8521-48a5aad5863b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:27.560266 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1217/2405] 10.222.0.26 () {58 vars in 1018 bytes} [Thu Jun 23 01:59:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:59:27.571010 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75b7023-560a-45c4-b276-a35b1c34c814 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:27.572995 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-d75b7023-560a-45c4-b276-a35b1c34c814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:27.587706 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75b7023-560a-45c4-b276-a35b1c34c814 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:27.593330 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8438cc25-e000-49e9-b269-201dbf072d44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:27.594296 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438cc25-e000-49e9-b269-201dbf072d44 service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:27.594646 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438cc25-e000-49e9-b269-201dbf072d44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:27.595126 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438cc25-e000-49e9-b269-201dbf072d44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:27.610584 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8438cc25-e000-49e9-b269-201dbf072d44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:27.612076 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8438cc25-e000-49e9-b269-201dbf072d44 service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:27.623470 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1189/2406] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:59:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:59:30.288212 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ddeadd-4d0a-402c-8061-41a8ed0d30d5 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:30.288695 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ddeadd-4d0a-402c-8061-41a8ed0d30d5 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:30.288789 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ddeadd-4d0a-402c-8061-41a8ed0d30d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:30.308047 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-f8ddeadd-4d0a-402c-8061-41a8ed0d30d5 None None] MFA Rules not processed for user `168e6181ad22460990f1f7361b483364`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:59:30.309316 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-f8ddeadd-4d0a-402c-8061-41a8ed0d30d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:30.364930 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1218/2407] 10.222.0.26 () {58 vars in 1009 bytes} [Thu Jun 23 01:59:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:59:30.387343 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8438cc25-e000-49e9-b269-201dbf072d44 service neutron] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:30.389567 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-8438cc25-e000-49e9-b269-201dbf072d44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:30.471418 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8438cc25-e000-49e9-b269-201dbf072d44 service neutron] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:30.476474 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dace5482-2c73-48cb-9684-fdd1cd5082f4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28e59688485241c4b6d51f99bbb3d059', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:30.477141 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace5482-2c73-48cb-9684-fdd1cd5082f4 service swift] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:30.477424 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace5482-2c73-48cb-9684-fdd1cd5082f4 service swift] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:30.477700 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace5482-2c73-48cb-9684-fdd1cd5082f4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:30.486645 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dace5482-2c73-48cb-9684-fdd1cd5082f4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:30.488258 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dace5482-2c73-48cb-9684-fdd1cd5082f4 service swift] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:30.496435 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1190/2408] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Jun 23 01:59:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 01:59:40.357590 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=91 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:59:40.493254 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=85 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 01:59:42.430860 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ddeadd-4d0a-402c-8061-41a8ed0d30d5 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:42.432420 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-f8ddeadd-4d0a-402c-8061-41a8ed0d30d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:42.554934 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ddeadd-4d0a-402c-8061-41a8ed0d30d5 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:42.560796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564aac9f-c390-4439-bda7-a1dd3657b1de service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5bf3baee1b843fa89b65e5485f6bebb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:42.561439 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564aac9f-c390-4439-bda7-a1dd3657b1de service placement] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:42.561714 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564aac9f-c390-4439-bda7-a1dd3657b1de service placement] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:42.562018 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564aac9f-c390-4439-bda7-a1dd3657b1de service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:42.563884 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-564aac9f-c390-4439-bda7-a1dd3657b1de service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:42.573445 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06863c6c-d928-4bd0-b215-d7bfa61d8ce1 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:42.573919 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06863c6c-d928-4bd0-b215-d7bfa61d8ce1 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:42.574029 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06863c6c-d928-4bd0-b215-d7bfa61d8ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:42.576410 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564aac9f-c390-4439-bda7-a1dd3657b1de service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:42.578122 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564aac9f-c390-4439-bda7-a1dd3657b1de service placement] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:42.589465 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1219/2409] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 01:59:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 01:59:42.594560 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-06863c6c-d928-4bd0-b215-d7bfa61d8ce1 None None] MFA Rules not processed for user `c14a81624a264ab280d46d063ae676ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:59:42.596004 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-06863c6c-d928-4bd0-b215-d7bfa61d8ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:42.671668 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1191/2410] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 01:59:42 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 01:59:52.222230 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde7aa3-98ba-43d8-9790-c19170a9b6d8 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:52.222713 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde7aa3-98ba-43d8-9790-c19170a9b6d8 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:52.222823 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde7aa3-98ba-43d8-9790-c19170a9b6d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:52.274190 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-fbde7aa3-98ba-43d8-9790-c19170a9b6d8 None None] MFA Rules not processed for user `168e6181ad22460990f1f7361b483364`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 01:59:52.275471 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-fbde7aa3-98ba-43d8-9790-c19170a9b6d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 01:59:52.295464 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1220/2411] 10.222.0.26 () {58 vars in 967 bytes} [Thu Jun 23 01:59:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 01:59:52.308604 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06863c6c-d928-4bd0-b215-d7bfa61d8ce1 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 01:59:52.331245 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06863c6c-d928-4bd0-b215-d7bfa61d8ce1 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 01:59:52.336810 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740c77ed-24e0-4b29-b377-758cb8202b29 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28e59688485241c4b6d51f99bbb3d059', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 01:59:52.337445 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c77ed-24e0-4b29-b377-758cb8202b29 service swift] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 01:59:52.337721 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c77ed-24e0-4b29-b377-758cb8202b29 service swift] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 01:59:52.337991 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c77ed-24e0-4b29-b377-758cb8202b29 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 01:59:52.346422 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740c77ed-24e0-4b29-b377-758cb8202b29 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 01:59:52.348048 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740c77ed-24e0-4b29-b377-758cb8202b29 service swift] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 01:59:52.356203 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1192/2412] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Jun 23 01:59:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 02:00:00.545138 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbde7aa3-98ba-43d8-9790-c19170a9b6d8 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:00:00.555596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbde7aa3-98ba-43d8-9790-c19170a9b6d8 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:00:00.560123 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91abd960-f0fb-405c-8f67-7df009bac36a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:00:00.560786 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91abd960-f0fb-405c-8f67-7df009bac36a service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:00:00.561065 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91abd960-f0fb-405c-8f67-7df009bac36a service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:00:00.561319 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91abd960-f0fb-405c-8f67-7df009bac36a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:00:00.563793 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-91abd960-f0fb-405c-8f67-7df009bac36a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:00:00.634006 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91abd960-f0fb-405c-8f67-7df009bac36a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:00:00.635488 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91abd960-f0fb-405c-8f67-7df009bac36a service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:00:00.645403 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1221/2413] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:00:00 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:00:02.353075 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=35 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:00:10.640761 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=89 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:00:43.603818 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3336b2-0a9d-4a40-984f-bf244bb4fd07 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:00:43.604492 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3336b2-0a9d-4a40-984f-bf244bb4fd07 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:00:43.604492 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3336b2-0a9d-4a40-984f-bf244bb4fd07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:00:43.622983 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-0e3336b2-0a9d-4a40-984f-bf244bb4fd07 None None] MFA Rules not processed for user `b2138120a6814dc4935a03203d35cb6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:00:43.624390 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-0e3336b2-0a9d-4a40-984f-bf244bb4fd07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:00:43.681507 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1193/2414] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:00:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:00:53.674500 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=25 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:00:54.000336 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f71356-663a-4fbc-b922-83b482232d5f None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:00:54.000870 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f71356-663a-4fbc-b922-83b482232d5f None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:00:54.000870 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f71356-663a-4fbc-b922-83b482232d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:00:54.019000 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-05f71356-663a-4fbc-b922-83b482232d5f None None] MFA Rules not processed for user `0f92d1dfc6c74b51aec7212f9a244a0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:00:54.020336 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-05f71356-663a-4fbc-b922-83b482232d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:00:54.075972 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1222/2415] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:00:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:01:04.069046 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=25 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:01:04.579506 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8706c765-7a91-4d20-8bfc-f9eee2c33cb0 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:01:04.580080 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8706c765-7a91-4d20-8bfc-f9eee2c33cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:01:04.580080 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8706c765-7a91-4d20-8bfc-f9eee2c33cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:01:04.599132 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-8706c765-7a91-4d20-8bfc-f9eee2c33cb0 None None] MFA Rules not processed for user `0287bd1260d24e5a94c00367b23c7a84`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:01:04.600472 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-8706c765-7a91-4d20-8bfc-f9eee2c33cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:01:04.658734 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1194/2416] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:01:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:01:14.651515 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=25 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:01:24.491020 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f71356-663a-4fbc-b922-83b482232d5f None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:01:24.501612 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f71356-663a-4fbc-b922-83b482232d5f None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:01:24.506144 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a12d7-3b1e-4194-b441-8fa94dd7eb30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:01:24.506988 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a12d7-3b1e-4194-b441-8fa94dd7eb30 service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:01:24.507927 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a12d7-3b1e-4194-b441-8fa94dd7eb30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:01:24.508237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a12d7-3b1e-4194-b441-8fa94dd7eb30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:01:24.510440 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-436a12d7-3b1e-4194-b441-8fa94dd7eb30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:01:24.526583 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436a12d7-3b1e-4194-b441-8fa94dd7eb30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:01:24.528063 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436a12d7-3b1e-4194-b441-8fa94dd7eb30 service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:01:24.539636 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1223/2417] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:01:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:01:26.163871 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8ad84-bda5-423f-a5c2-62431d9b92f7 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:01:26.168122 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8ad84-bda5-423f-a5c2-62431d9b92f7 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:01:26.168523 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8ad84-bda5-423f-a5c2-62431d9b92f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:01:26.199812 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-f0d8ad84-bda5-423f-a5c2-62431d9b92f7 None None] MFA Rules not processed for user `168e6181ad22460990f1f7361b483364`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:01:26.201135 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-f0d8ad84-bda5-423f-a5c2-62431d9b92f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:01:26.217337 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1195/2418] 10.222.0.26 () {58 vars in 967 bytes} [Thu Jun 23 02:01:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:01:26.304929 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a12d7-3b1e-4194-b441-8fa94dd7eb30 service neutron] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:01:26.320930 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a12d7-3b1e-4194-b441-8fa94dd7eb30 service neutron] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:01:26.326424 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414132f8-efc3-4bf7-9980-5621c82d9201 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28e59688485241c4b6d51f99bbb3d059', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:01:26.328054 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414132f8-efc3-4bf7-9980-5621c82d9201 service swift] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:01:26.328441 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414132f8-efc3-4bf7-9980-5621c82d9201 service swift] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:01:26.328810 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414132f8-efc3-4bf7-9980-5621c82d9201 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:01:26.339060 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414132f8-efc3-4bf7-9980-5621c82d9201 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:01:26.340821 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414132f8-efc3-4bf7-9980-5621c82d9201 service swift] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:01:26.350567 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1224/2419] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Jun 23 02:01:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 02:01:28.865755 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85f87d-595e-4174-b49b-d072ca15f0e5 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:01:28.866232 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85f87d-595e-4174-b49b-d072ca15f0e5 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:01:28.866763 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85f87d-595e-4174-b49b-d072ca15f0e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:01:28.886837 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-5f85f87d-595e-4174-b49b-d072ca15f0e5 None None] MFA Rules not processed for user `ababea3e976f46889f2b1980edfee883`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:01:28.888511 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-5f85f87d-595e-4174-b49b-d072ca15f0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:01:28.952966 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1196/2420] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:01:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:01:28.999188 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414132f8-efc3-4bf7-9980-5621c82d9201 service swift] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:01:29.021546 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414132f8-efc3-4bf7-9980-5621c82d9201 service swift] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:01:29.026806 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a80c1f5-0f7c-457f-93f5-59d7c98ab64a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c081b4cf34146ca8eb197b62a133433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:01:29.027863 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80c1f5-0f7c-457f-93f5-59d7c98ab64a service glance] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:01:29.028359 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80c1f5-0f7c-457f-93f5-59d7c98ab64a service glance] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:01:29.028623 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80c1f5-0f7c-457f-93f5-59d7c98ab64a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:01:29.030836 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-6a80c1f5-0f7c-457f-93f5-59d7c98ab64a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:01:29.045233 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a80c1f5-0f7c-457f-93f5-59d7c98ab64a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:01:29.046969 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a80c1f5-0f7c-457f-93f5-59d7c98ab64a service glance] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:01:29.060897 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1225/2421] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:01:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:01:35.094044 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f85f87d-595e-4174-b49b-d072ca15f0e5 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:01:35.095608 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-5f85f87d-595e-4174-b49b-d072ca15f0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:01:35.114161 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f85f87d-595e-4174-b49b-d072ca15f0e5 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:01:35.122875 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02968481-7bc0-46a4-af57-00f1b44b6f36 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5bf3baee1b843fa89b65e5485f6bebb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:01:35.123571 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02968481-7bc0-46a4-af57-00f1b44b6f36 service placement] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:01:35.123850 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02968481-7bc0-46a4-af57-00f1b44b6f36 service placement] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:01:35.124122 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02968481-7bc0-46a4-af57-00f1b44b6f36 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:01:35.126079 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-02968481-7bc0-46a4-af57-00f1b44b6f36 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:01:35.138992 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02968481-7bc0-46a4-af57-00f1b44b6f36 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:01:35.141153 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02968481-7bc0-46a4-af57-00f1b44b6f36 service placement] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:01:35.154697 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1197/2422] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:01:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:01:35.190197 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a80c1f5-0f7c-457f-93f5-59d7c98ab64a service glance] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:01:35.192052 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-6a80c1f5-0f7c-457f-93f5-59d7c98ab64a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:01:35.276984 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a80c1f5-0f7c-457f-93f5-59d7c98ab64a service glance] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:01:35.281639 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339bf977-69c1-4f17-9ec3-805a3f02a221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:01:35.282598 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339bf977-69c1-4f17-9ec3-805a3f02a221 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:01:35.282920 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339bf977-69c1-4f17-9ec3-805a3f02a221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:01:35.283183 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339bf977-69c1-4f17-9ec3-805a3f02a221 admin admin] PATH_INFO: `/v3/users/3d72a03c4eec4412856ee48cb967c46d` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:01:35.284752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339bf977-69c1-4f17-9ec3-805a3f02a221 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d72a03c4eec4412856ee48cb967c46d)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:01:35.286725 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339bf977-69c1-4f17-9ec3-805a3f02a221 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:01:35.344319 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-339bf977-69c1-4f17-9ec3-805a3f02a221 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d72a03c4eec4412856ee48cb967c46d'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:01:35.362721 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-339bf977-69c1-4f17-9ec3-805a3f02a221 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d72a03c4eec4412856ee48cb967c46d'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:01:35.371982 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-339bf977-69c1-4f17-9ec3-805a3f02a221 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d72a03c4eec4412856ee48cb967c46d'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:01:35.380842 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-339bf977-69c1-4f17-9ec3-805a3f02a221 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d72a03c4eec4412856ee48cb967c46d'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:01:35.381538 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-339bf977-69c1-4f17-9ec3-805a3f02a221 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d72a03c4eec4412856ee48cb967c46d'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:01:35.387830 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1226/2423] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 02:01:35 2022] DELETE /identity/v3/users/3d72a03c4eec4412856ee48cb967c46d => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:01:35.583406 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02968481-7bc0-46a4-af57-00f1b44b6f36 service placement] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:01:35.595794 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02968481-7bc0-46a4-af57-00f1b44b6f36 service placement] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:01:35.655695 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee0a8b5-6326-4599-b612-5cd5384d67b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:01:35.656567 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0a8b5-6326-4599-b612-5cd5384d67b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:01:35.656850 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0a8b5-6326-4599-b612-5cd5384d67b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:01:35.657145 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0a8b5-6326-4599-b612-5cd5384d67b2 admin admin] PATH_INFO: `/v3/projects/5a7db67320b04863ac7d28369425dd19` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:01:35.658341 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee0a8b5-6326-4599-b612-5cd5384d67b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a7db67320b04863ac7d28369425dd19)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:01:35.660318 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee0a8b5-6326-4599-b612-5cd5384d67b2 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:01:35.708312 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-fee0a8b5-6326-4599-b612-5cd5384d67b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a7db67320b04863ac7d28369425dd19', 'request_id': 'req-fee0a8b5-6326-4599-b612-5cd5384d67b2'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:01:35.714578 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-fee0a8b5-6326-4599-b612-5cd5384d67b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a7db67320b04863ac7d28369425dd19', 'request_id': 'req-fee0a8b5-6326-4599-b612-5cd5384d67b2'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:01:35.723927 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-fee0a8b5-6326-4599-b612-5cd5384d67b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a7db67320b04863ac7d28369425dd19', 'request_id': 'req-fee0a8b5-6326-4599-b612-5cd5384d67b2'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:01:35.735072 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-fee0a8b5-6326-4599-b612-5cd5384d67b2 admin admin] The token cache is being invalidate because project 5a7db67320b04863ac7d28369425dd19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 02:01:35.735436 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-fee0a8b5-6326-4599-b612-5cd5384d67b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:01:35.736170 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-fee0a8b5-6326-4599-b612-5cd5384d67b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:01:35.737740 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1198/2424] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 02:01:35 2022] DELETE /identity/v3/projects/5a7db67320b04863ac7d28369425dd19 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:01:35.742220 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339bf977-69c1-4f17-9ec3-805a3f02a221 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:01:35.743673 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-339bf977-69c1-4f17-9ec3-805a3f02a221 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:01:35.757712 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339bf977-69c1-4f17-9ec3-805a3f02a221 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:01:35.791068 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:01:35.821270 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e54e35b-f040-44ae-bb8c-1dfec320b6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:01:35.822189 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54e35b-f040-44ae-bb8c-1dfec320b6c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:01:35.822471 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54e35b-f040-44ae-bb8c-1dfec320b6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:01:35.822745 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54e35b-f040-44ae-bb8c-1dfec320b6c8 admin admin] PATH_INFO: `/v3/users/f92ee0300da141d3b8a1aec100e8c0d6` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:01:35.824006 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e54e35b-f040-44ae-bb8c-1dfec320b6c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f92ee0300da141d3b8a1aec100e8c0d6)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:01:35.826021 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e54e35b-f040-44ae-bb8c-1dfec320b6c8 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:01:35.883583 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-4e54e35b-f040-44ae-bb8c-1dfec320b6c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f92ee0300da141d3b8a1aec100e8c0d6'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:01:35.896741 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-4e54e35b-f040-44ae-bb8c-1dfec320b6c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f92ee0300da141d3b8a1aec100e8c0d6'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:01:35.906945 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-4e54e35b-f040-44ae-bb8c-1dfec320b6c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f92ee0300da141d3b8a1aec100e8c0d6'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:01:35.917234 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-4e54e35b-f040-44ae-bb8c-1dfec320b6c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f92ee0300da141d3b8a1aec100e8c0d6'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:01:35.918175 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-4e54e35b-f040-44ae-bb8c-1dfec320b6c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f92ee0300da141d3b8a1aec100e8c0d6'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:01:35.926131 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1227/2425] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 02:01:35 2022] DELETE /identity/v3/users/f92ee0300da141d3b8a1aec100e8c0d6 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:01:36.105989 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee0a8b5-6326-4599-b612-5cd5384d67b2 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:01:36.107717 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-fee0a8b5-6326-4599-b612-5cd5384d67b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:01:36.121679 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee0a8b5-6326-4599-b612-5cd5384d67b2 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:01:36.136936 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:01:36.188028 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ddfec4-6d0a-4f42-b2cb-2675f6162191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:01:36.189336 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ddfec4-6d0a-4f42-b2cb-2675f6162191 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:01:36.189458 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ddfec4-6d0a-4f42-b2cb-2675f6162191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:01:36.189618 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ddfec4-6d0a-4f42-b2cb-2675f6162191 admin admin] PATH_INFO: `/v3/projects/5d85388598234c5e88414a1dbf17a5f8` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:01:36.190809 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ddfec4-6d0a-4f42-b2cb-2675f6162191 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d85388598234c5e88414a1dbf17a5f8)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:01:36.192760 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ddfec4-6d0a-4f42-b2cb-2675f6162191 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:01:36.241343 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-83ddfec4-6d0a-4f42-b2cb-2675f6162191 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d85388598234c5e88414a1dbf17a5f8', 'request_id': 'req-83ddfec4-6d0a-4f42-b2cb-2675f6162191'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:01:36.248352 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-83ddfec4-6d0a-4f42-b2cb-2675f6162191 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d85388598234c5e88414a1dbf17a5f8', 'request_id': 'req-83ddfec4-6d0a-4f42-b2cb-2675f6162191'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:01:36.256867 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-83ddfec4-6d0a-4f42-b2cb-2675f6162191 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d85388598234c5e88414a1dbf17a5f8', 'request_id': 'req-83ddfec4-6d0a-4f42-b2cb-2675f6162191'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:01:36.267419 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-83ddfec4-6d0a-4f42-b2cb-2675f6162191 admin admin] The token cache is being invalidate because project 5d85388598234c5e88414a1dbf17a5f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 02:01:36.267707 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-83ddfec4-6d0a-4f42-b2cb-2675f6162191 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:01:36.268342 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-83ddfec4-6d0a-4f42-b2cb-2675f6162191 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:01:36.269924 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1199/2426] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 02:01:36 2022] DELETE /identity/v3/projects/5d85388598234c5e88414a1dbf17a5f8 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:01:40.595244 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e54e35b-f040-44ae-bb8c-1dfec320b6c8 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:01:40.596939 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-4e54e35b-f040-44ae-bb8c-1dfec320b6c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:01:40.691408 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e54e35b-f040-44ae-bb8c-1dfec320b6c8 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:01:40.736568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:01:40.743272 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f13b5-5fa2-4df0-a8b0-56311fa3f4c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:01:40.743272 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f13b5-5fa2-4df0-a8b0-56311fa3f4c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:01:40.743599 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f13b5-5fa2-4df0-a8b0-56311fa3f4c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:01:40.743886 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f13b5-5fa2-4df0-a8b0-56311fa3f4c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:01:40.746428 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-d48f13b5-5fa2-4df0-a8b0-56311fa3f4c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:01:40.813933 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f13b5-5fa2-4df0-a8b0-56311fa3f4c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:01:40.815775 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f13b5-5fa2-4df0-a8b0-56311fa3f4c7 service cinder] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:01:40.825633 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1228/2427] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:01:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:01:46.264373 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=44,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:01:50.820984 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=27 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:01:51.070695 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ddfec4-6d0a-4f42-b2cb-2675f6162191 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:01:51.072308 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-83ddfec4-6d0a-4f42-b2cb-2675f6162191 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:01:51.144719 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ddfec4-6d0a-4f42-b2cb-2675f6162191 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:01:51.149461 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f46c0a-ce41-4b8d-91b5-590fb8d015fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:01:51.150588 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f46c0a-ce41-4b8d-91b5-590fb8d015fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:01:51.150876 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f46c0a-ce41-4b8d-91b5-590fb8d015fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:01:51.151166 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f46c0a-ce41-4b8d-91b5-590fb8d015fe admin admin] PATH_INFO: `/v3/users/b2138120a6814dc4935a03203d35cb6d` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:01:51.152458 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f46c0a-ce41-4b8d-91b5-590fb8d015fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2138120a6814dc4935a03203d35cb6d)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:01:51.154863 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f46c0a-ce41-4b8d-91b5-590fb8d015fe admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:01:51.207077 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-86f46c0a-ce41-4b8d-91b5-590fb8d015fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2138120a6814dc4935a03203d35cb6d'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:01:51.219236 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-86f46c0a-ce41-4b8d-91b5-590fb8d015fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2138120a6814dc4935a03203d35cb6d'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:01:51.228392 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-86f46c0a-ce41-4b8d-91b5-590fb8d015fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2138120a6814dc4935a03203d35cb6d'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:01:51.236717 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-86f46c0a-ce41-4b8d-91b5-590fb8d015fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2138120a6814dc4935a03203d35cb6d'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:01:51.237336 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-86f46c0a-ce41-4b8d-91b5-590fb8d015fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2138120a6814dc4935a03203d35cb6d'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:01:51.244126 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1200/2428] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 02:01:51 2022] DELETE /identity/v3/users/b2138120a6814dc4935a03203d35cb6d => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:01:51.423402 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f13b5-5fa2-4df0-a8b0-56311fa3f4c7 service cinder] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:01:51.438225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f13b5-5fa2-4df0-a8b0-56311fa3f4c7 service cinder] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:01:51.502862 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c273a58d-dc8b-4acb-a374-b93fdfc7b378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:01:51.503786 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273a58d-dc8b-4acb-a374-b93fdfc7b378 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:01:51.504135 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273a58d-dc8b-4acb-a374-b93fdfc7b378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:01:51.504408 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273a58d-dc8b-4acb-a374-b93fdfc7b378 admin admin] PATH_INFO: `/v3/projects/9b84d8e675cc4386bb229f124b399e03` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:01:51.505588 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c273a58d-dc8b-4acb-a374-b93fdfc7b378 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b84d8e675cc4386bb229f124b399e03)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:01:51.507547 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c273a58d-dc8b-4acb-a374-b93fdfc7b378 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:01:51.557801 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-c273a58d-dc8b-4acb-a374-b93fdfc7b378 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b84d8e675cc4386bb229f124b399e03', 'request_id': 'req-c273a58d-dc8b-4acb-a374-b93fdfc7b378'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:01:51.563971 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-c273a58d-dc8b-4acb-a374-b93fdfc7b378 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b84d8e675cc4386bb229f124b399e03', 'request_id': 'req-c273a58d-dc8b-4acb-a374-b93fdfc7b378'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:01:51.572500 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-c273a58d-dc8b-4acb-a374-b93fdfc7b378 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b84d8e675cc4386bb229f124b399e03', 'request_id': 'req-c273a58d-dc8b-4acb-a374-b93fdfc7b378'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:01:51.582765 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-c273a58d-dc8b-4acb-a374-b93fdfc7b378 admin admin] The token cache is being invalidate because project 9b84d8e675cc4386bb229f124b399e03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 02:01:51.583033 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-c273a58d-dc8b-4acb-a374-b93fdfc7b378 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:01:51.583712 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-c273a58d-dc8b-4acb-a374-b93fdfc7b378 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:01:51.585433 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1229/2429] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 02:01:51 2022] DELETE /identity/v3/projects/9b84d8e675cc4386bb229f124b399e03 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:02:00.811007 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f46c0a-ce41-4b8d-91b5-590fb8d015fe admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:00.812726 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-86f46c0a-ce41-4b8d-91b5-590fb8d015fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:00.882795 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f46c0a-ce41-4b8d-91b5-590fb8d015fe admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:00.916896 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:02:00.935579 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25812ec-45c7-4793-ad90-7dd6424ff634 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:00.936228 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25812ec-45c7-4793-ad90-7dd6424ff634 service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:00.936504 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25812ec-45c7-4793-ad90-7dd6424ff634 service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:00.936775 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25812ec-45c7-4793-ad90-7dd6424ff634 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:00.938614 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-e25812ec-45c7-4793-ad90-7dd6424ff634 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:01.002189 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25812ec-45c7-4793-ad90-7dd6424ff634 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:01.003666 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25812ec-45c7-4793-ad90-7dd6424ff634 service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:01.013364 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1201/2430] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:02:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:02:01.579836 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:02:03.604856 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c273a58d-dc8b-4acb-a374-b93fdfc7b378 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:03.605961 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-c273a58d-dc8b-4acb-a374-b93fdfc7b378 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:03.686682 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c273a58d-dc8b-4acb-a374-b93fdfc7b378 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:03.693398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e543d2b-360e-4387-917d-a36bdf33beb5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4e83df33b64a3ebf28f54e2d94fe63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:03.694093 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e543d2b-360e-4387-917d-a36bdf33beb5 service nova] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:03.694384 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e543d2b-360e-4387-917d-a36bdf33beb5 service nova] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:03.694665 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e543d2b-360e-4387-917d-a36bdf33beb5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:03.696629 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-8e543d2b-360e-4387-917d-a36bdf33beb5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:03.763039 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e543d2b-360e-4387-917d-a36bdf33beb5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:03.764651 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e543d2b-360e-4387-917d-a36bdf33beb5 service nova] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:03.774447 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1230/2431] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:02:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:02:04.484040 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25812ec-45c7-4793-ad90-7dd6424ff634 service neutron] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:04.485525 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-e25812ec-45c7-4793-ad90-7dd6424ff634 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:04.498887 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25812ec-45c7-4793-ad90-7dd6424ff634 service neutron] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:04.503652 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad03242c-c6f2-4099-9bc2-c74eaf964328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:04.504539 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03242c-c6f2-4099-9bc2-c74eaf964328 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:04.504820 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03242c-c6f2-4099-9bc2-c74eaf964328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:04.505096 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03242c-c6f2-4099-9bc2-c74eaf964328 admin admin] PATH_INFO: `/v3/users/0f92d1dfc6c74b51aec7212f9a244a0a` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:04.506388 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad03242c-c6f2-4099-9bc2-c74eaf964328 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f92d1dfc6c74b51aec7212f9a244a0a)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:04.508373 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad03242c-c6f2-4099-9bc2-c74eaf964328 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:04.558058 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-ad03242c-c6f2-4099-9bc2-c74eaf964328 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f92d1dfc6c74b51aec7212f9a244a0a'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:04.570288 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-ad03242c-c6f2-4099-9bc2-c74eaf964328 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f92d1dfc6c74b51aec7212f9a244a0a'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:04.578851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-ad03242c-c6f2-4099-9bc2-c74eaf964328 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f92d1dfc6c74b51aec7212f9a244a0a'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:04.586835 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-ad03242c-c6f2-4099-9bc2-c74eaf964328 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f92d1dfc6c74b51aec7212f9a244a0a'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:04.587510 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-ad03242c-c6f2-4099-9bc2-c74eaf964328 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f92d1dfc6c74b51aec7212f9a244a0a'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:04.596842 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1202/2432] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 02:02:04 2022] DELETE /identity/v3/users/0f92d1dfc6c74b51aec7212f9a244a0a => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:02:04.784403 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e543d2b-360e-4387-917d-a36bdf33beb5 service nova] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:04.795686 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e543d2b-360e-4387-917d-a36bdf33beb5 service nova] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:04.854302 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf53ab-6898-4249-8882-3fd2e3202e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:04.855173 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf53ab-6898-4249-8882-3fd2e3202e91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:04.855485 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf53ab-6898-4249-8882-3fd2e3202e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:04.855762 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf53ab-6898-4249-8882-3fd2e3202e91 admin admin] PATH_INFO: `/v3/projects/e7608bb4bbf1435289dc08731d51df22` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:04.856994 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbf53ab-6898-4249-8882-3fd2e3202e91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7608bb4bbf1435289dc08731d51df22)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:04.858972 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbf53ab-6898-4249-8882-3fd2e3202e91 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:04.900893 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:02:04.910894 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-cfbf53ab-6898-4249-8882-3fd2e3202e91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7608bb4bbf1435289dc08731d51df22', 'request_id': 'req-cfbf53ab-6898-4249-8882-3fd2e3202e91'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:04.920674 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-cfbf53ab-6898-4249-8882-3fd2e3202e91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7608bb4bbf1435289dc08731d51df22', 'request_id': 'req-cfbf53ab-6898-4249-8882-3fd2e3202e91'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:04.929932 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-cfbf53ab-6898-4249-8882-3fd2e3202e91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7608bb4bbf1435289dc08731d51df22', 'request_id': 'req-cfbf53ab-6898-4249-8882-3fd2e3202e91'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:04.940890 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-cfbf53ab-6898-4249-8882-3fd2e3202e91 admin admin] The token cache is being invalidate because project e7608bb4bbf1435289dc08731d51df22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 02:02:04.941175 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-cfbf53ab-6898-4249-8882-3fd2e3202e91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:04.941799 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-cfbf53ab-6898-4249-8882-3fd2e3202e91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:04.943435 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1231/2433] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 02:02:04 2022] DELETE /identity/v3/projects/e7608bb4bbf1435289dc08731d51df22 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:02:04.949064 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c777d368-e699-445c-bd61-35468a9c3c74 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:04.949356 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c777d368-e699-445c-bd61-35468a9c3c74 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:04.949636 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c777d368-e699-445c-bd61-35468a9c3c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:04.967636 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-c777d368-e699-445c-bd61-35468a9c3c74 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:02:04.968864 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-c777d368-e699-445c-bd61-35468a9c3c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:05.019116 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:02:05.031286 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1203/2434] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:02:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:02:05.039001 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ebf53-4d78-47bc-bcec-f4571c49499b None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:05.039341 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ebf53-4d78-47bc-bcec-f4571c49499b None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:05.039623 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ebf53-4d78-47bc-bcec-f4571c49499b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:05.059590 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-bb1ebf53-4d78-47bc-bcec-f4571c49499b None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:02:05.060802 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-bb1ebf53-4d78-47bc-bcec-f4571c49499b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:05.069471 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1232/2435] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:02:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:02:05.073950 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c777d368-e699-445c-bd61-35468a9c3c74 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:05.084125 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c777d368-e699-445c-bd61-35468a9c3c74 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:05.089266 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5f83e6-c73c-4b69-b745-c62130274587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:05.090040 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f83e6-c73c-4b69-b745-c62130274587 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:05.090320 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f83e6-c73c-4b69-b745-c62130274587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:05.090619 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f83e6-c73c-4b69-b745-c62130274587 admin admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:05.091372 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5f83e6-c73c-4b69-b745-c62130274587 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 02:02:05.091681 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5f83e6-c73c-4b69-b745-c62130274587 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:05.093277 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5f83e6-c73c-4b69-b745-c62130274587 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:05.105104 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1204/2436] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 02:02:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 02:02:05.109386 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1ebf53-4d78-47bc-bcec-f4571c49499b None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:05.118730 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1ebf53-4d78-47bc-bcec-f4571c49499b None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:05.124798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bafcf10-1ba0-4a81-8d5f-815afa614c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:05.125451 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bafcf10-1ba0-4a81-8d5f-815afa614c42 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:05.125735 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bafcf10-1ba0-4a81-8d5f-815afa614c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:05.126010 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bafcf10-1ba0-4a81-8d5f-815afa614c42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:05.126714 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bafcf10-1ba0-4a81-8d5f-815afa614c42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:05.128690 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bafcf10-1ba0-4a81-8d5f-815afa614c42 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:05.159531 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1233/2437] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 02:02:05 2022] POST /identity/v3/projects => generated 367 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 02:02:05.164001 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5f83e6-c73c-4b69-b745-c62130274587 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:05.174660 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5f83e6-c73c-4b69-b745-c62130274587 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:05.234157 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4272811-3aba-426d-831c-fba1ef4bd193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:05.234867 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4272811-3aba-426d-831c-fba1ef4bd193 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:05.235153 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4272811-3aba-426d-831c-fba1ef4bd193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:05.235463 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4272811-3aba-426d-831c-fba1ef4bd193 admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:05.236162 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4272811-3aba-426d-831c-fba1ef4bd193 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:05.238291 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4272811-3aba-426d-831c-fba1ef4bd193 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:05.263865 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1205/2438] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 02:02:05 2022] POST /identity/v3/users => generated 284 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:02:05.268295 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bafcf10-1ba0-4a81-8d5f-815afa614c42 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:05.279199 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bafcf10-1ba0-4a81-8d5f-815afa614c42 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:05.284656 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6f0d44-ab2e-471a-8e88-1558418aa050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:05.285346 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f0d44-ab2e-471a-8e88-1558418aa050 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:05.285642 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f0d44-ab2e-471a-8e88-1558418aa050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:05.285875 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f0d44-ab2e-471a-8e88-1558418aa050 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:05.286628 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6f0d44-ab2e-471a-8e88-1558418aa050 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:05.288427 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6f0d44-ab2e-471a-8e88-1558418aa050 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:05.299983 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1234/2439] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:02:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:02:05.304202 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4272811-3aba-426d-831c-fba1ef4bd193 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:05.314795 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4272811-3aba-426d-831c-fba1ef4bd193 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:05.320325 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bece2e-6623-4d9d-bc82-ce4d5bb18a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:05.320990 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bece2e-6623-4d9d-bc82-ce4d5bb18a8e admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:05.321305 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bece2e-6623-4d9d-bc82-ce4d5bb18a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:05.321578 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bece2e-6623-4d9d-bc82-ce4d5bb18a8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:05.322315 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bece2e-6623-4d9d-bc82-ce4d5bb18a8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:05.323968 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bece2e-6623-4d9d-bc82-ce4d5bb18a8e admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:05.334044 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1206/2440] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:02:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:02:05.338377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6f0d44-ab2e-471a-8e88-1558418aa050 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:05.348451 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6f0d44-ab2e-471a-8e88-1558418aa050 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:05.354267 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee9da23-b0ca-418d-b7e1-46105f14c035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:05.355192 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9da23-b0ca-418d-b7e1-46105f14c035 admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:05.355506 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9da23-b0ca-418d-b7e1-46105f14c035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:05.355784 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9da23-b0ca-418d-b7e1-46105f14c035 admin admin] PATH_INFO: `/v3/projects/6665a5806b534c3889f4788dcffc35c6/users/e8c4f3f96ab042a2af9b6ad7e437095f/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:05.370910 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee9da23-b0ca-418d-b7e1-46105f14c035 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6665a5806b534c3889f4788dcffc35c6, user_id=e8c4f3f96ab042a2af9b6ad7e437095f, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:05.373684 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee9da23-b0ca-418d-b7e1-46105f14c035 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:05.388105 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1235/2441] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 02:02:05 2022] PUT /identity/v3/projects/6665a5806b534c3889f4788dcffc35c6/users/e8c4f3f96ab042a2af9b6ad7e437095f/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 02:02:05.398291 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509897f-a23f-43e1-aeaf-315e01acac50 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:05.398576 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509897f-a23f-43e1-aeaf-315e01acac50 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:05.398855 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509897f-a23f-43e1-aeaf-315e01acac50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:05.426163 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-d509897f-a23f-43e1-aeaf-315e01acac50 None None] MFA Rules not processed for user `e8c4f3f96ab042a2af9b6ad7e437095f`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:02:05.427432 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-d509897f-a23f-43e1-aeaf-315e01acac50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:05.491873 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1207/2442] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:02:05 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:02:05.496554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee9da23-b0ca-418d-b7e1-46105f14c035 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:05.562110 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee9da23-b0ca-418d-b7e1-46105f14c035 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:05.567368 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c30ad5b-763e-4372-8e13-f73d49edea53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:05.568017 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30ad5b-763e-4372-8e13-f73d49edea53 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:05.568279 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30ad5b-763e-4372-8e13-f73d49edea53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:05.568566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30ad5b-763e-4372-8e13-f73d49edea53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:05.569275 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c30ad5b-763e-4372-8e13-f73d49edea53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:05.571198 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c30ad5b-763e-4372-8e13-f73d49edea53 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:05.599638 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1236/2443] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 02:02:05 2022] POST /identity/v3/projects => generated 367 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:02:05.603970 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d509897f-a23f-43e1-aeaf-315e01acac50 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:05.614822 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d509897f-a23f-43e1-aeaf-315e01acac50 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:05.667858 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:02:05.678878 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce9b20-d8ef-4c70-af77-b07b402e668a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:05.679530 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce9b20-d8ef-4c70-af77-b07b402e668a admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:05.679814 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce9b20-d8ef-4c70-af77-b07b402e668a admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:05.680091 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce9b20-d8ef-4c70-af77-b07b402e668a admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:05.680853 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ce9b20-d8ef-4c70-af77-b07b402e668a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:05.682816 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ce9b20-d8ef-4c70-af77-b07b402e668a admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:05.704775 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1208/2444] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 02:02:05 2022] POST /identity/v3/users => generated 290 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:02:05.709047 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c30ad5b-763e-4372-8e13-f73d49edea53 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:05.719825 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c30ad5b-763e-4372-8e13-f73d49edea53 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:05.725412 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698aa10d-dbda-4df3-a1ac-676c56a510b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:05.726070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698aa10d-dbda-4df3-a1ac-676c56a510b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:05.726351 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698aa10d-dbda-4df3-a1ac-676c56a510b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:05.726624 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698aa10d-dbda-4df3-a1ac-676c56a510b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:05.727404 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698aa10d-dbda-4df3-a1ac-676c56a510b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:05.729031 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698aa10d-dbda-4df3-a1ac-676c56a510b0 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:05.739922 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1237/2445] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:02:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:02:05.744316 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce9b20-d8ef-4c70-af77-b07b402e668a admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:05.755350 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce9b20-d8ef-4c70-af77-b07b402e668a admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:05.761091 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758b34d-1033-4c3c-9496-f3589536f49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:05.761995 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758b34d-1033-4c3c-9496-f3589536f49b admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:05.762273 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758b34d-1033-4c3c-9496-f3589536f49b admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:05.762554 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758b34d-1033-4c3c-9496-f3589536f49b admin admin] PATH_INFO: `/v3/projects/f64565ca30ab4a5abce69cef63c72f13/users/4b8ad3659e824b8bb0c99f015a39bb83/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:05.778939 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1758b34d-1033-4c3c-9496-f3589536f49b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f64565ca30ab4a5abce69cef63c72f13, user_id=4b8ad3659e824b8bb0c99f015a39bb83, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:05.781664 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1758b34d-1033-4c3c-9496-f3589536f49b admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:05.797825 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1209/2446] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 02:02:05 2022] PUT /identity/v3/projects/f64565ca30ab4a5abce69cef63c72f13/users/4b8ad3659e824b8bb0c99f015a39bb83/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:02:05.801969 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698aa10d-dbda-4df3-a1ac-676c56a510b0 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:05.813169 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698aa10d-dbda-4df3-a1ac-676c56a510b0 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:05.854323 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:02:05.879249 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ec0a9-dd3d-423a-a5ad-4f7733eab8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:05.879926 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ec0a9-dd3d-423a-a5ad-4f7733eab8b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:05.880208 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ec0a9-dd3d-423a-a5ad-4f7733eab8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:05.880483 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ec0a9-dd3d-423a-a5ad-4f7733eab8b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:05.881287 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439ec0a9-dd3d-423a-a5ad-4f7733eab8b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:05.882903 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439ec0a9-dd3d-423a-a5ad-4f7733eab8b7 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:05.892776 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1238/2447] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:02:05 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:02:05.898551 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758b34d-1033-4c3c-9496-f3589536f49b admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:05.909169 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758b34d-1033-4c3c-9496-f3589536f49b admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:05.914563 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5937f45-c712-420b-8ded-0d321bbc6e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:05.915487 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5937f45-c712-420b-8ded-0d321bbc6e3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:05.915802 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5937f45-c712-420b-8ded-0d321bbc6e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:05.916079 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5937f45-c712-420b-8ded-0d321bbc6e3e admin admin] PATH_INFO: `/v3/projects/f64565ca30ab4a5abce69cef63c72f13/users/4b8ad3659e824b8bb0c99f015a39bb83/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:05.918503 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5937f45-c712-420b-8ded-0d321bbc6e3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f64565ca30ab4a5abce69cef63c72f13, user_id=4b8ad3659e824b8bb0c99f015a39bb83, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:05.921224 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5937f45-c712-420b-8ded-0d321bbc6e3e admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:05.932589 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1210/2448] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 02:02:05 2022] PUT /identity/v3/projects/f64565ca30ab4a5abce69cef63c72f13/users/4b8ad3659e824b8bb0c99f015a39bb83/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 02:02:05.943898 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f327b66b-220b-40d2-84a9-f651f3a54502 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:05.944195 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f327b66b-220b-40d2-84a9-f651f3a54502 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:05.944466 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f327b66b-220b-40d2-84a9-f651f3a54502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:05.977048 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-f327b66b-220b-40d2-84a9-f651f3a54502 None None] MFA Rules not processed for user `4b8ad3659e824b8bb0c99f015a39bb83`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:02:05.978262 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-f327b66b-220b-40d2-84a9-f651f3a54502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:06.051655 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1239/2449] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:02:05 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:02:06.062834 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5937f45-c712-420b-8ded-0d321bbc6e3e admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:06.064794 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-c5937f45-c712-420b-8ded-0d321bbc6e3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:06.126158 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5937f45-c712-420b-8ded-0d321bbc6e3e admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:06.130211 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6b44af-e4ad-4936-82a3-3405e485a3d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:06.130917 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b44af-e4ad-4936-82a3-3405e485a3d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:06.131201 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b44af-e4ad-4936-82a3-3405e485a3d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:06.131503 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b44af-e4ad-4936-82a3-3405e485a3d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:06.142294 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6b44af-e4ad-4936-82a3-3405e485a3d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:06.144004 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6b44af-e4ad-4936-82a3-3405e485a3d2 service cinder] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:06.153722 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1211/2450] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:02:06 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:02:07.495824 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f327b66b-220b-40d2-84a9-f651f3a54502 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:07.563310 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f327b66b-220b-40d2-84a9-f651f3a54502 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:07.568439 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588057cc-442e-46aa-b75e-21f9b15b8821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:07.569324 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588057cc-442e-46aa-b75e-21f9b15b8821 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:07.569604 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588057cc-442e-46aa-b75e-21f9b15b8821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:07.569882 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588057cc-442e-46aa-b75e-21f9b15b8821 admin admin] PATH_INFO: `/v3/users/e8c4f3f96ab042a2af9b6ad7e437095f` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:07.571146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588057cc-442e-46aa-b75e-21f9b15b8821 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8c4f3f96ab042a2af9b6ad7e437095f)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:07.573143 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588057cc-442e-46aa-b75e-21f9b15b8821 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:07.621525 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-588057cc-442e-46aa-b75e-21f9b15b8821 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8c4f3f96ab042a2af9b6ad7e437095f'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:07.631647 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:02:07.634934 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-588057cc-442e-46aa-b75e-21f9b15b8821 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8c4f3f96ab042a2af9b6ad7e437095f'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:07.643530 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-588057cc-442e-46aa-b75e-21f9b15b8821 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8c4f3f96ab042a2af9b6ad7e437095f'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:07.650522 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-588057cc-442e-46aa-b75e-21f9b15b8821 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8c4f3f96ab042a2af9b6ad7e437095f'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:07.651145 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-588057cc-442e-46aa-b75e-21f9b15b8821 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8c4f3f96ab042a2af9b6ad7e437095f'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:07.657024 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1240/2451] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 02:02:07 2022] DELETE /identity/v3/users/e8c4f3f96ab042a2af9b6ad7e437095f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:02:07.661537 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5611ccc-29b1-4038-8d7c-3ba40cf208f1 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:07.662123 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5611ccc-29b1-4038-8d7c-3ba40cf208f1 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:07.662222 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5611ccc-29b1-4038-8d7c-3ba40cf208f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:07.681057 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-d5611ccc-29b1-4038-8d7c-3ba40cf208f1 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:02:07.682285 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-d5611ccc-29b1-4038-8d7c-3ba40cf208f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:07.746036 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1212/2452] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:02:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:02:07.757481 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588057cc-442e-46aa-b75e-21f9b15b8821 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:07.759365 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-588057cc-442e-46aa-b75e-21f9b15b8821 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:07.825555 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588057cc-442e-46aa-b75e-21f9b15b8821 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:07.829845 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca62bd-6d6b-459e-bd21-77d05695589b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:07.830483 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca62bd-6d6b-459e-bd21-77d05695589b service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:07.830761 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca62bd-6d6b-459e-bd21-77d05695589b service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:07.831032 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca62bd-6d6b-459e-bd21-77d05695589b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:07.840629 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eca62bd-6d6b-459e-bd21-77d05695589b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:07.842066 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eca62bd-6d6b-459e-bd21-77d05695589b service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:07.851832 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1241/2453] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:02:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:02:08.207594 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5611ccc-29b1-4038-8d7c-3ba40cf208f1 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:08.209038 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-d5611ccc-29b1-4038-8d7c-3ba40cf208f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:08.223123 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5611ccc-29b1-4038-8d7c-3ba40cf208f1 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:08.228520 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2d9889-b54c-40bc-95b6-f73d8d692998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:08.229401 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d9889-b54c-40bc-95b6-f73d8d692998 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:08.229679 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d9889-b54c-40bc-95b6-f73d8d692998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:08.229964 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d9889-b54c-40bc-95b6-f73d8d692998 admin admin] PATH_INFO: `/v3/projects/6665a5806b534c3889f4788dcffc35c6` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:08.231252 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2d9889-b54c-40bc-95b6-f73d8d692998 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6665a5806b534c3889f4788dcffc35c6)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:08.233198 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2d9889-b54c-40bc-95b6-f73d8d692998 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:08.280357 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:02:08.285947 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-8b2d9889-b54c-40bc-95b6-f73d8d692998 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6665a5806b534c3889f4788dcffc35c6', 'request_id': 'req-8b2d9889-b54c-40bc-95b6-f73d8d692998'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:08.295072 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-8b2d9889-b54c-40bc-95b6-f73d8d692998 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6665a5806b534c3889f4788dcffc35c6', 'request_id': 'req-8b2d9889-b54c-40bc-95b6-f73d8d692998'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:08.304960 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-8b2d9889-b54c-40bc-95b6-f73d8d692998 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6665a5806b534c3889f4788dcffc35c6', 'request_id': 'req-8b2d9889-b54c-40bc-95b6-f73d8d692998'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:08.315658 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-8b2d9889-b54c-40bc-95b6-f73d8d692998 admin admin] The token cache is being invalidate because project 6665a5806b534c3889f4788dcffc35c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 02:02:08.315947 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-8b2d9889-b54c-40bc-95b6-f73d8d692998 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:08.316618 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-8b2d9889-b54c-40bc-95b6-f73d8d692998 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:08.318399 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1213/2454] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 02:02:08 2022] DELETE /identity/v3/projects/6665a5806b534c3889f4788dcffc35c6 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:02:08.322468 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca62bd-6d6b-459e-bd21-77d05695589b service neutron] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:08.332156 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca62bd-6d6b-459e-bd21-77d05695589b service neutron] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:08.388372 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f1f2d-5d14-4834-8035-09ea5db95528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:08.389244 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f1f2d-5d14-4834-8035-09ea5db95528 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:08.389518 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f1f2d-5d14-4834-8035-09ea5db95528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:08.389789 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f1f2d-5d14-4834-8035-09ea5db95528 admin admin] PATH_INFO: `/v3/users/4b8ad3659e824b8bb0c99f015a39bb83` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:08.391019 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f1f2d-5d14-4834-8035-09ea5db95528 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b8ad3659e824b8bb0c99f015a39bb83)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:08.393032 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f1f2d-5d14-4834-8035-09ea5db95528 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:08.442428 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-330f1f2d-5d14-4834-8035-09ea5db95528 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b8ad3659e824b8bb0c99f015a39bb83'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:08.454452 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-330f1f2d-5d14-4834-8035-09ea5db95528 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b8ad3659e824b8bb0c99f015a39bb83'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:08.462353 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-330f1f2d-5d14-4834-8035-09ea5db95528 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b8ad3659e824b8bb0c99f015a39bb83'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:08.469136 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-330f1f2d-5d14-4834-8035-09ea5db95528 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b8ad3659e824b8bb0c99f015a39bb83'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:08.469746 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-330f1f2d-5d14-4834-8035-09ea5db95528 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b8ad3659e824b8bb0c99f015a39bb83'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:08.471837 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:02:08.476607 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1242/2455] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 02:02:08 2022] DELETE /identity/v3/users/4b8ad3659e824b8bb0c99f015a39bb83 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:02:08.753552 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2d9889-b54c-40bc-95b6-f73d8d692998 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:08.755093 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-8b2d9889-b54c-40bc-95b6-f73d8d692998 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:08.769233 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2d9889-b54c-40bc-95b6-f73d8d692998 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:08.830740 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a5ed77-2ea9-4267-a4da-5e147826926b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:08.831557 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5ed77-2ea9-4267-a4da-5e147826926b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:08.831835 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5ed77-2ea9-4267-a4da-5e147826926b admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:08.832107 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5ed77-2ea9-4267-a4da-5e147826926b admin admin] PATH_INFO: `/v3/projects/f64565ca30ab4a5abce69cef63c72f13` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:08.833305 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a5ed77-2ea9-4267-a4da-5e147826926b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f64565ca30ab4a5abce69cef63c72f13)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:08.835376 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a5ed77-2ea9-4267-a4da-5e147826926b admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:08.888437 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-39a5ed77-2ea9-4267-a4da-5e147826926b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f64565ca30ab4a5abce69cef63c72f13', 'request_id': 'req-39a5ed77-2ea9-4267-a4da-5e147826926b'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:08.894497 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-39a5ed77-2ea9-4267-a4da-5e147826926b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f64565ca30ab4a5abce69cef63c72f13', 'request_id': 'req-39a5ed77-2ea9-4267-a4da-5e147826926b'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:08.903195 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-39a5ed77-2ea9-4267-a4da-5e147826926b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f64565ca30ab4a5abce69cef63c72f13', 'request_id': 'req-39a5ed77-2ea9-4267-a4da-5e147826926b'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:08.913765 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-39a5ed77-2ea9-4267-a4da-5e147826926b admin admin] The token cache is being invalidate because project f64565ca30ab4a5abce69cef63c72f13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 02:02:08.914048 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-39a5ed77-2ea9-4267-a4da-5e147826926b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:08.914673 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-39a5ed77-2ea9-4267-a4da-5e147826926b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:08.916285 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1214/2456] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 02:02:08 2022] DELETE /identity/v3/projects/f64565ca30ab4a5abce69cef63c72f13 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:02:08.921663 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe0ef0-48d5-4c54-bb32-ae5f1152bee8 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:08.922127 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe0ef0-48d5-4c54-bb32-ae5f1152bee8 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:08.922252 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe0ef0-48d5-4c54-bb32-ae5f1152bee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:08.940103 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-bcfe0ef0-48d5-4c54-bb32-ae5f1152bee8 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:02:08.941297 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-bcfe0ef0-48d5-4c54-bb32-ae5f1152bee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:09.009906 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1243/2457] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:02:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:02:09.017948 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a574a49-800c-4f78-96d4-a132a087db88 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:09.018228 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a574a49-800c-4f78-96d4-a132a087db88 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:09.018503 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a574a49-800c-4f78-96d4-a132a087db88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:09.037122 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-5a574a49-800c-4f78-96d4-a132a087db88 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:02:09.038321 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-5a574a49-800c-4f78-96d4-a132a087db88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:09.052508 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1215/2458] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:02:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:02:09.059414 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe0ef0-48d5-4c54-bb32-ae5f1152bee8 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:09.074431 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe0ef0-48d5-4c54-bb32-ae5f1152bee8 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:09.082144 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42057321-5376-4243-9c0e-23169a11055d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:09.083115 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42057321-5376-4243-9c0e-23169a11055d admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:09.083655 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42057321-5376-4243-9c0e-23169a11055d admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:09.085019 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42057321-5376-4243-9c0e-23169a11055d admin admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:09.086298 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42057321-5376-4243-9c0e-23169a11055d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 02:02:09.086772 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42057321-5376-4243-9c0e-23169a11055d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:09.089527 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42057321-5376-4243-9c0e-23169a11055d admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:09.108923 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1244/2459] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 02:02:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 02:02:09.113364 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a574a49-800c-4f78-96d4-a132a087db88 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:09.124100 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a574a49-800c-4f78-96d4-a132a087db88 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:09.129570 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb67cd01-01ab-4586-bece-7b13d997cd43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:09.130333 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb67cd01-01ab-4586-bece-7b13d997cd43 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:09.130659 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb67cd01-01ab-4586-bece-7b13d997cd43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:09.130935 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb67cd01-01ab-4586-bece-7b13d997cd43 admin admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:09.131666 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb67cd01-01ab-4586-bece-7b13d997cd43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:09.133557 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb67cd01-01ab-4586-bece-7b13d997cd43 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:09.162223 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1216/2460] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 02:02:09 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:02:09.166820 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42057321-5376-4243-9c0e-23169a11055d admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:09.177741 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42057321-5376-4243-9c0e-23169a11055d admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:09.236818 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9051e1-9560-4069-9072-f44368d6072b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:09.237495 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9051e1-9560-4069-9072-f44368d6072b admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:09.237777 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9051e1-9560-4069-9072-f44368d6072b admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:09.238054 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9051e1-9560-4069-9072-f44368d6072b admin admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:09.238745 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9051e1-9560-4069-9072-f44368d6072b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:09.240798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9051e1-9560-4069-9072-f44368d6072b admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:09.264580 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1245/2461] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 02:02:09 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:02:09.269281 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb67cd01-01ab-4586-bece-7b13d997cd43 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:09.281304 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb67cd01-01ab-4586-bece-7b13d997cd43 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:09.288147 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2971d405-2646-4d72-bc48-62cd017f7ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:09.288870 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2971d405-2646-4d72-bc48-62cd017f7ff5 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:09.289149 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2971d405-2646-4d72-bc48-62cd017f7ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:09.289425 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2971d405-2646-4d72-bc48-62cd017f7ff5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:09.290199 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2971d405-2646-4d72-bc48-62cd017f7ff5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:09.291853 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2971d405-2646-4d72-bc48-62cd017f7ff5 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:09.305352 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1217/2462] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:02:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:02:09.309901 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9051e1-9560-4069-9072-f44368d6072b admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:09.320789 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9051e1-9560-4069-9072-f44368d6072b admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:09.326834 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c22805f-5a43-49f5-9a43-dfa42667253b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:09.327540 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c22805f-5a43-49f5-9a43-dfa42667253b admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:09.327818 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c22805f-5a43-49f5-9a43-dfa42667253b admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:09.328175 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c22805f-5a43-49f5-9a43-dfa42667253b admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:09.328912 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c22805f-5a43-49f5-9a43-dfa42667253b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:09.330536 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c22805f-5a43-49f5-9a43-dfa42667253b admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:09.341565 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1246/2463] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:02:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:02:09.345840 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2971d405-2646-4d72-bc48-62cd017f7ff5 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:09.356840 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2971d405-2646-4d72-bc48-62cd017f7ff5 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:09.361736 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b690b4-6ff6-4f64-a0a3-b68e83a81939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:09.362614 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b690b4-6ff6-4f64-a0a3-b68e83a81939 admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:09.362891 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b690b4-6ff6-4f64-a0a3-b68e83a81939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:09.363161 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b690b4-6ff6-4f64-a0a3-b68e83a81939 admin admin] PATH_INFO: `/v3/projects/26d4e7e965244ad5874cf1a0b38869a6/users/c0f6e5669f174732823b168f792a422c/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:09.379718 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b690b4-6ff6-4f64-a0a3-b68e83a81939 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26d4e7e965244ad5874cf1a0b38869a6, user_id=c0f6e5669f174732823b168f792a422c, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:09.383937 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b690b4-6ff6-4f64-a0a3-b68e83a81939 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:09.396374 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:02:09.403291 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1218/2464] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 02:02:09 2022] PUT /identity/v3/projects/26d4e7e965244ad5874cf1a0b38869a6/users/c0f6e5669f174732823b168f792a422c/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:02:09.414032 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf6bc1-3521-4072-ab46-c4ed2487a38b None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:09.414320 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf6bc1-3521-4072-ab46-c4ed2487a38b None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:09.414623 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf6bc1-3521-4072-ab46-c4ed2487a38b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:09.441509 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-cfbf6bc1-3521-4072-ab46-c4ed2487a38b None None] MFA Rules not processed for user `c0f6e5669f174732823b168f792a422c`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:02:09.442727 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-cfbf6bc1-3521-4072-ab46-c4ed2487a38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:09.498607 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:02:09.508644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1247/2465] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:02:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:02:09.513120 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b690b4-6ff6-4f64-a0a3-b68e83a81939 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:09.577995 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b690b4-6ff6-4f64-a0a3-b68e83a81939 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:09.583092 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79bb898-97c1-4671-af1b-ab91ecbe523b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:09.583885 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79bb898-97c1-4671-af1b-ab91ecbe523b admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:09.584169 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79bb898-97c1-4671-af1b-ab91ecbe523b admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:09.584443 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79bb898-97c1-4671-af1b-ab91ecbe523b admin admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:09.585151 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79bb898-97c1-4671-af1b-ab91ecbe523b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:09.587072 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79bb898-97c1-4671-af1b-ab91ecbe523b admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:09.612703 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1219/2466] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 02:02:09 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:02:09.616957 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf6bc1-3521-4072-ab46-c4ed2487a38b None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:09.627844 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf6bc1-3521-4072-ab46-c4ed2487a38b None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:09.687025 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b3c5bd-cfaf-4fb3-a381-c7f4901ef9e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:09.687739 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3c5bd-cfaf-4fb3-a381-c7f4901ef9e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:09.688016 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3c5bd-cfaf-4fb3-a381-c7f4901ef9e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:09.688285 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3c5bd-cfaf-4fb3-a381-c7f4901ef9e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:09.688999 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b3c5bd-cfaf-4fb3-a381-c7f4901ef9e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:09.690928 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b3c5bd-cfaf-4fb3-a381-c7f4901ef9e2 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:09.711833 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1248/2467] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 02:02:09 2022] POST /identity/v3/users => generated 299 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:02:09.715945 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79bb898-97c1-4671-af1b-ab91ecbe523b admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:09.726949 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79bb898-97c1-4671-af1b-ab91ecbe523b admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:09.732221 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f995f5-3c45-4f25-887d-d116c6bdbe1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:09.732907 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f995f5-3c45-4f25-887d-d116c6bdbe1f admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:09.733189 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f995f5-3c45-4f25-887d-d116c6bdbe1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:09.733462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f995f5-3c45-4f25-887d-d116c6bdbe1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:09.734194 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f995f5-3c45-4f25-887d-d116c6bdbe1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:09.735871 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f995f5-3c45-4f25-887d-d116c6bdbe1f admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:09.746544 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1220/2468] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:02:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:02:09.750522 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b3c5bd-cfaf-4fb3-a381-c7f4901ef9e2 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:09.761268 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b3c5bd-cfaf-4fb3-a381-c7f4901ef9e2 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:09.767015 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a6f3e-6fb3-4036-83c4-68e0bc0828c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:09.767926 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a6f3e-6fb3-4036-83c4-68e0bc0828c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:09.768204 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a6f3e-6fb3-4036-83c4-68e0bc0828c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:09.768475 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a6f3e-6fb3-4036-83c4-68e0bc0828c2 admin admin] PATH_INFO: `/v3/projects/4965996a7b804d598e3f85e085b8edd1/users/50cc9106a69a4561af88b3b73e3d1717/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:09.784295 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422a6f3e-6fb3-4036-83c4-68e0bc0828c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4965996a7b804d598e3f85e085b8edd1, user_id=50cc9106a69a4561af88b3b73e3d1717, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:09.787046 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422a6f3e-6fb3-4036-83c4-68e0bc0828c2 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:09.804609 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1249/2469] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 02:02:09 2022] PUT /identity/v3/projects/4965996a7b804d598e3f85e085b8edd1/users/50cc9106a69a4561af88b3b73e3d1717/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:02:09.810083 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f995f5-3c45-4f25-887d-d116c6bdbe1f admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:09.822293 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f995f5-3c45-4f25-887d-d116c6bdbe1f admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:09.883661 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ccb66-f727-41a9-ad56-b3ccc766c910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:09.884264 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ccb66-f727-41a9-ad56-b3ccc766c910 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:09.884548 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ccb66-f727-41a9-ad56-b3ccc766c910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:09.884823 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ccb66-f727-41a9-ad56-b3ccc766c910 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:09.885558 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6ccb66-f727-41a9-ad56-b3ccc766c910 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:09.887168 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6ccb66-f727-41a9-ad56-b3ccc766c910 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:09.896852 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1221/2470] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:02:09 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:02:09.901262 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a6f3e-6fb3-4036-83c4-68e0bc0828c2 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:09.911835 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a6f3e-6fb3-4036-83c4-68e0bc0828c2 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:09.916858 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896c000f-268e-4e54-81d3-0f2edf002133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:09.917739 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896c000f-268e-4e54-81d3-0f2edf002133 admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:09.918936 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896c000f-268e-4e54-81d3-0f2edf002133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:09.919247 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896c000f-268e-4e54-81d3-0f2edf002133 admin admin] PATH_INFO: `/v3/projects/4965996a7b804d598e3f85e085b8edd1/users/50cc9106a69a4561af88b3b73e3d1717/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:09.921528 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896c000f-268e-4e54-81d3-0f2edf002133 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4965996a7b804d598e3f85e085b8edd1, user_id=50cc9106a69a4561af88b3b73e3d1717, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:09.924434 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896c000f-268e-4e54-81d3-0f2edf002133 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:09.935535 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1250/2471] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 02:02:09 2022] PUT /identity/v3/projects/4965996a7b804d598e3f85e085b8edd1/users/50cc9106a69a4561af88b3b73e3d1717/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:02:09.951252 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715c186-64e4-4de2-84db-b1ae026bad76 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:09.951568 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715c186-64e4-4de2-84db-b1ae026bad76 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:09.951849 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715c186-64e4-4de2-84db-b1ae026bad76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:09.982715 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-8715c186-64e4-4de2-84db-b1ae026bad76 None None] MFA Rules not processed for user `50cc9106a69a4561af88b3b73e3d1717`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:02:09.983973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-8715c186-64e4-4de2-84db-b1ae026bad76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:10.045541 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:02:10.056924 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1222/2472] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:02:09 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:02:10.068704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896c000f-268e-4e54-81d3-0f2edf002133 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:10.070599 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-896c000f-268e-4e54-81d3-0f2edf002133 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:10.131287 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896c000f-268e-4e54-81d3-0f2edf002133 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:10.135814 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7be3b02-3fe6-46db-8db0-0f3a1fd8483d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:10.136319 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7be3b02-3fe6-46db-8db0-0f3a1fd8483d service cinder] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:10.136645 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7be3b02-3fe6-46db-8db0-0f3a1fd8483d service cinder] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:10.137012 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7be3b02-3fe6-46db-8db0-0f3a1fd8483d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:10.146529 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7be3b02-3fe6-46db-8db0-0f3a1fd8483d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:10.148027 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7be3b02-3fe6-46db-8db0-0f3a1fd8483d service cinder] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:10.157610 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1251/2473] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:02:10 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 02:02:10.222199 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8715c186-64e4-4de2-84db-b1ae026bad76 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:10.288928 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8715c186-64e4-4de2-84db-b1ae026bad76 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:10.294788 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f884bd7-8dfb-46fe-a668-c0f7f330cca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:10.296020 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f884bd7-8dfb-46fe-a668-c0f7f330cca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:10.296378 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f884bd7-8dfb-46fe-a668-c0f7f330cca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:10.296555 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f884bd7-8dfb-46fe-a668-c0f7f330cca5 admin admin] PATH_INFO: `/v3/users/c0f6e5669f174732823b168f792a422c` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:10.297804 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f884bd7-8dfb-46fe-a668-c0f7f330cca5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0f6e5669f174732823b168f792a422c)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:10.299780 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f884bd7-8dfb-46fe-a668-c0f7f330cca5 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:10.348855 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-5f884bd7-8dfb-46fe-a668-c0f7f330cca5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0f6e5669f174732823b168f792a422c'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:10.362595 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-5f884bd7-8dfb-46fe-a668-c0f7f330cca5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0f6e5669f174732823b168f792a422c'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:10.371096 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-5f884bd7-8dfb-46fe-a668-c0f7f330cca5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0f6e5669f174732823b168f792a422c'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:10.378988 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-5f884bd7-8dfb-46fe-a668-c0f7f330cca5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0f6e5669f174732823b168f792a422c'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:10.379695 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-5f884bd7-8dfb-46fe-a668-c0f7f330cca5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0f6e5669f174732823b168f792a422c'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:10.386062 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1223/2474] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 02:02:10 2022] DELETE /identity/v3/users/c0f6e5669f174732823b168f792a422c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:02:10.390589 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb06452-51b0-4073-92d1-25c4a1f8a287 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:10.390909 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb06452-51b0-4073-92d1-25c4a1f8a287 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:10.391182 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb06452-51b0-4073-92d1-25c4a1f8a287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:10.413649 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-3bb06452-51b0-4073-92d1-25c4a1f8a287 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:02:10.414866 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-3bb06452-51b0-4073-92d1-25c4a1f8a287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:10.463259 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:02:10.480236 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1252/2475] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:02:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:02:10.490342 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f884bd7-8dfb-46fe-a668-c0f7f330cca5 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:10.491895 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-5f884bd7-8dfb-46fe-a668-c0f7f330cca5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:10.552881 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f884bd7-8dfb-46fe-a668-c0f7f330cca5 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:10.557150 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40d94fc-624b-405a-9336-a109b49e7e64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:10.557809 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d94fc-624b-405a-9336-a109b49e7e64 service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:10.558099 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d94fc-624b-405a-9336-a109b49e7e64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:10.558371 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d94fc-624b-405a-9336-a109b49e7e64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:10.567962 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40d94fc-624b-405a-9336-a109b49e7e64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:10.569438 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40d94fc-624b-405a-9336-a109b49e7e64 service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:10.579304 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1224/2476] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:02:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 02:02:10.862528 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb06452-51b0-4073-92d1-25c4a1f8a287 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:10.864130 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-3bb06452-51b0-4073-92d1-25c4a1f8a287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:10.876211 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb06452-51b0-4073-92d1-25c4a1f8a287 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:10.882038 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354aec9e-4a6c-49e6-a556-6baafe581094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:10.882928 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354aec9e-4a6c-49e6-a556-6baafe581094 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:10.883229 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354aec9e-4a6c-49e6-a556-6baafe581094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:10.883506 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354aec9e-4a6c-49e6-a556-6baafe581094 admin admin] PATH_INFO: `/v3/projects/26d4e7e965244ad5874cf1a0b38869a6` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:10.884774 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354aec9e-4a6c-49e6-a556-6baafe581094 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26d4e7e965244ad5874cf1a0b38869a6)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:10.886684 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354aec9e-4a6c-49e6-a556-6baafe581094 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:10.934581 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-354aec9e-4a6c-49e6-a556-6baafe581094 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26d4e7e965244ad5874cf1a0b38869a6', 'request_id': 'req-354aec9e-4a6c-49e6-a556-6baafe581094'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:10.940886 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-354aec9e-4a6c-49e6-a556-6baafe581094 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26d4e7e965244ad5874cf1a0b38869a6', 'request_id': 'req-354aec9e-4a6c-49e6-a556-6baafe581094'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:10.953492 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-354aec9e-4a6c-49e6-a556-6baafe581094 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26d4e7e965244ad5874cf1a0b38869a6', 'request_id': 'req-354aec9e-4a6c-49e6-a556-6baafe581094'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:10.965746 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-354aec9e-4a6c-49e6-a556-6baafe581094 admin admin] The token cache is being invalidate because project 26d4e7e965244ad5874cf1a0b38869a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 02:02:10.966027 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-354aec9e-4a6c-49e6-a556-6baafe581094 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:10.966709 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-354aec9e-4a6c-49e6-a556-6baafe581094 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:10.968522 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1253/2477] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 02:02:10 2022] DELETE /identity/v3/projects/26d4e7e965244ad5874cf1a0b38869a6 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:02:10.972639 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40d94fc-624b-405a-9336-a109b49e7e64 service neutron] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:10.983304 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40d94fc-624b-405a-9336-a109b49e7e64 service neutron] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:10.994616 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:02:11.046125 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b342db3-475f-4232-8aba-7770fbd5c6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:11.047377 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b342db3-475f-4232-8aba-7770fbd5c6ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:11.047786 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b342db3-475f-4232-8aba-7770fbd5c6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:11.048169 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b342db3-475f-4232-8aba-7770fbd5c6ae admin admin] PATH_INFO: `/v3/users/50cc9106a69a4561af88b3b73e3d1717` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:11.050144 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b342db3-475f-4232-8aba-7770fbd5c6ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=50cc9106a69a4561af88b3b73e3d1717)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:11.052960 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b342db3-475f-4232-8aba-7770fbd5c6ae admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:11.098814 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-9b342db3-475f-4232-8aba-7770fbd5c6ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50cc9106a69a4561af88b3b73e3d1717'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:11.110505 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-9b342db3-475f-4232-8aba-7770fbd5c6ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50cc9106a69a4561af88b3b73e3d1717'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:11.118107 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-9b342db3-475f-4232-8aba-7770fbd5c6ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50cc9106a69a4561af88b3b73e3d1717'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:11.125085 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-9b342db3-475f-4232-8aba-7770fbd5c6ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50cc9106a69a4561af88b3b73e3d1717'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:11.125753 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-9b342db3-475f-4232-8aba-7770fbd5c6ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50cc9106a69a4561af88b3b73e3d1717'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:11.131706 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1225/2478] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 02:02:10 2022] DELETE /identity/v3/users/50cc9106a69a4561af88b3b73e3d1717 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:02:11.437891 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354aec9e-4a6c-49e6-a556-6baafe581094 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:11.439756 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-354aec9e-4a6c-49e6-a556-6baafe581094 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:11.455670 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354aec9e-4a6c-49e6-a556-6baafe581094 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:11.517484 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce8934f-d3ee-4e64-8963-852a9ca407b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:11.518260 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8934f-d3ee-4e64-8963-852a9ca407b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:11.518538 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8934f-d3ee-4e64-8963-852a9ca407b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:11.519528 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8934f-d3ee-4e64-8963-852a9ca407b6 admin admin] PATH_INFO: `/v3/projects/4965996a7b804d598e3f85e085b8edd1` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:11.520064 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce8934f-d3ee-4e64-8963-852a9ca407b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4965996a7b804d598e3f85e085b8edd1)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:11.522743 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce8934f-d3ee-4e64-8963-852a9ca407b6 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:11.570754 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-9ce8934f-d3ee-4e64-8963-852a9ca407b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4965996a7b804d598e3f85e085b8edd1', 'request_id': 'req-9ce8934f-d3ee-4e64-8963-852a9ca407b6'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:11.576941 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-9ce8934f-d3ee-4e64-8963-852a9ca407b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4965996a7b804d598e3f85e085b8edd1', 'request_id': 'req-9ce8934f-d3ee-4e64-8963-852a9ca407b6'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:11.585922 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-9ce8934f-d3ee-4e64-8963-852a9ca407b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4965996a7b804d598e3f85e085b8edd1', 'request_id': 'req-9ce8934f-d3ee-4e64-8963-852a9ca407b6'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:11.596545 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-9ce8934f-d3ee-4e64-8963-852a9ca407b6 admin admin] The token cache is being invalidate because project 4965996a7b804d598e3f85e085b8edd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 02:02:11.596840 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-9ce8934f-d3ee-4e64-8963-852a9ca407b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:11.597485 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-9ce8934f-d3ee-4e64-8963-852a9ca407b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:11.599139 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1254/2479] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 02:02:11 2022] DELETE /identity/v3/projects/4965996a7b804d598e3f85e085b8edd1 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:02:11.604225 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf6495-3c8c-4dda-973e-080e3507641c None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:11.604572 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf6495-3c8c-4dda-973e-080e3507641c None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:11.604862 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf6495-3c8c-4dda-973e-080e3507641c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:11.623227 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-ecaf6495-3c8c-4dda-973e-080e3507641c None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:02:11.624471 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-ecaf6495-3c8c-4dda-973e-080e3507641c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:11.686827 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1226/2480] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:02:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:02:11.694977 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb8e4f-9a12-4c0e-8fdf-0b0ed6aceee9 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:11.695312 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb8e4f-9a12-4c0e-8fdf-0b0ed6aceee9 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:11.695594 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb8e4f-9a12-4c0e-8fdf-0b0ed6aceee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:11.703620 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:02:11.716501 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-dfcb8e4f-9a12-4c0e-8fdf-0b0ed6aceee9 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:02:11.718292 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-dfcb8e4f-9a12-4c0e-8fdf-0b0ed6aceee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:11.727292 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1255/2481] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:02:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:02:11.731626 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf6495-3c8c-4dda-973e-080e3507641c None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:11.741627 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf6495-3c8c-4dda-973e-080e3507641c None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:11.746602 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3612985-4538-4066-a3f4-f4eb89680e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:11.747285 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3612985-4538-4066-a3f4-f4eb89680e0e admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:11.747561 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3612985-4538-4066-a3f4-f4eb89680e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:11.747855 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3612985-4538-4066-a3f4-f4eb89680e0e admin admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:11.748583 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3612985-4538-4066-a3f4-f4eb89680e0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 02:02:11.748887 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3612985-4538-4066-a3f4-f4eb89680e0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:11.750482 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3612985-4538-4066-a3f4-f4eb89680e0e admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:11.762172 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1227/2482] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 02:02:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 02:02:11.766305 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb8e4f-9a12-4c0e-8fdf-0b0ed6aceee9 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:11.776073 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb8e4f-9a12-4c0e-8fdf-0b0ed6aceee9 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:11.782191 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f98f3-831d-432a-85e4-5d21fe836469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:11.782857 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f98f3-831d-432a-85e4-5d21fe836469 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:11.783136 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f98f3-831d-432a-85e4-5d21fe836469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:11.783438 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f98f3-831d-432a-85e4-5d21fe836469 admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:11.784134 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76f98f3-831d-432a-85e4-5d21fe836469 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:11.786058 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76f98f3-831d-432a-85e4-5d21fe836469 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:11.821456 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1256/2483] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 02:02:11 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:02:11.825387 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3612985-4538-4066-a3f4-f4eb89680e0e admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:11.838209 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3612985-4538-4066-a3f4-f4eb89680e0e admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:11.859432 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:02:11.901675 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a9d231-0d42-444a-8c6b-f9ddc8260e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:11.902346 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a9d231-0d42-444a-8c6b-f9ddc8260e90 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:11.902624 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a9d231-0d42-444a-8c6b-f9ddc8260e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:11.902913 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a9d231-0d42-444a-8c6b-f9ddc8260e90 admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:11.903702 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a9d231-0d42-444a-8c6b-f9ddc8260e90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:11.905625 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a9d231-0d42-444a-8c6b-f9ddc8260e90 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:11.930433 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1228/2484] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 02:02:11 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:02:11.933809 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f98f3-831d-432a-85e4-5d21fe836469 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:11.948059 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f98f3-831d-432a-85e4-5d21fe836469 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:11.953631 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d35d96-bf56-49e4-a7d3-719f927066b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:11.954293 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d35d96-bf56-49e4-a7d3-719f927066b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:11.954604 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d35d96-bf56-49e4-a7d3-719f927066b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:11.954878 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d35d96-bf56-49e4-a7d3-719f927066b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:11.955670 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d35d96-bf56-49e4-a7d3-719f927066b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:11.957315 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d35d96-bf56-49e4-a7d3-719f927066b9 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:11.968412 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1257/2485] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:02:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:02:11.972479 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a9d231-0d42-444a-8c6b-f9ddc8260e90 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:11.982589 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a9d231-0d42-444a-8c6b-f9ddc8260e90 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:11.988221 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5540e65e-f427-498d-8237-b28d40bd9147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:11.988882 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540e65e-f427-498d-8237-b28d40bd9147 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:11.989196 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540e65e-f427-498d-8237-b28d40bd9147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:11.989472 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540e65e-f427-498d-8237-b28d40bd9147 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:11.990206 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5540e65e-f427-498d-8237-b28d40bd9147 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:11.991865 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5540e65e-f427-498d-8237-b28d40bd9147 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:12.002076 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1229/2486] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:02:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:02:12.006166 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d35d96-bf56-49e4-a7d3-719f927066b9 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:12.016596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d35d96-bf56-49e4-a7d3-719f927066b9 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:12.022140 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deda76c8-7056-4b21-a45e-46da7518f30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:12.023021 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda76c8-7056-4b21-a45e-46da7518f30a admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:12.023325 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda76c8-7056-4b21-a45e-46da7518f30a admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:12.023598 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda76c8-7056-4b21-a45e-46da7518f30a admin admin] PATH_INFO: `/v3/projects/586e177d2f7341d985de3551a3e55697/users/460b6517023c46578e9f0ff7e690351d/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:12.038703 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deda76c8-7056-4b21-a45e-46da7518f30a admin admin] RBAC: Authorizing `identity:create_grant(project_id=586e177d2f7341d985de3551a3e55697, user_id=460b6517023c46578e9f0ff7e690351d, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:12.041439 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deda76c8-7056-4b21-a45e-46da7518f30a admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:12.061646 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1258/2487] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 02:02:12 2022] PUT /identity/v3/projects/586e177d2f7341d985de3551a3e55697/users/460b6517023c46578e9f0ff7e690351d/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:02:12.074162 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bdcc6f-5e5f-4a05-8165-a81eb9c68685 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:12.074448 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bdcc6f-5e5f-4a05-8165-a81eb9c68685 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:12.074724 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bdcc6f-5e5f-4a05-8165-a81eb9c68685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:12.102806 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-48bdcc6f-5e5f-4a05-8165-a81eb9c68685 None None] MFA Rules not processed for user `460b6517023c46578e9f0ff7e690351d`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:02:12.104042 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-48bdcc6f-5e5f-4a05-8165-a81eb9c68685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:12.173788 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1230/2488] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:02:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:02:12.184948 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deda76c8-7056-4b21-a45e-46da7518f30a admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:12.187012 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-deda76c8-7056-4b21-a45e-46da7518f30a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:12.258074 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deda76c8-7056-4b21-a45e-46da7518f30a admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:12.262236 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf7f045-b8a8-412c-8805-197b88edee12 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c081b4cf34146ca8eb197b62a133433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:12.262862 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7f045-b8a8-412c-8805-197b88edee12 service glance] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:12.263149 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7f045-b8a8-412c-8805-197b88edee12 service glance] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:12.263464 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7f045-b8a8-412c-8805-197b88edee12 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:12.274830 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf7f045-b8a8-412c-8805-197b88edee12 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:12.276354 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf7f045-b8a8-412c-8805-197b88edee12 service glance] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:12.285639 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1259/2489] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:02:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:02:22.170498 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=56,keystone:INSERT=3 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:02:22.281185 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=56,keystone:INSERT=2 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:02:26.587557 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bdcc6f-5e5f-4a05-8165-a81eb9c68685 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:26.594971 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-48bdcc6f-5e5f-4a05-8165-a81eb9c68685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:26.730171 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bdcc6f-5e5f-4a05-8165-a81eb9c68685 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:26.742274 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4093de-76b8-4c21-809a-5dbc1fdb3023 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4e83df33b64a3ebf28f54e2d94fe63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:26.743016 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4093de-76b8-4c21-809a-5dbc1fdb3023 service nova] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:26.743395 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4093de-76b8-4c21-809a-5dbc1fdb3023 service nova] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:26.743736 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4093de-76b8-4c21-809a-5dbc1fdb3023 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:26.745925 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-bd4093de-76b8-4c21-809a-5dbc1fdb3023 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:26.761166 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4093de-76b8-4c21-809a-5dbc1fdb3023 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:26.764345 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4093de-76b8-4c21-809a-5dbc1fdb3023 service nova] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:26.795515 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1231/2490] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:02:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:02:27.544742 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a2c9d-b71e-479d-a2d8-18521f11c22c None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:27.545202 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a2c9d-b71e-479d-a2d8-18521f11c22c None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:27.545300 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a2c9d-b71e-479d-a2d8-18521f11c22c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:27.564950 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-412a2c9d-b71e-479d-a2d8-18521f11c22c None None] MFA Rules not processed for user `c14a81624a264ab280d46d063ae676ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:02:27.566199 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-412a2c9d-b71e-479d-a2d8-18521f11c22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:27.630106 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1260/2491] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:02:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:02:27.875783 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4093de-76b8-4c21-809a-5dbc1fdb3023 service nova] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:27.943826 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4093de-76b8-4c21-809a-5dbc1fdb3023 service nova] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:27.946527 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412a2c9d-b71e-479d-a2d8-18521f11c22c None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:27.947981 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-412a2c9d-b71e-479d-a2d8-18521f11c22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:27.948546 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e6c05e-0545-4c24-9f47-fd5bf5b6ee20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:27.949548 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6c05e-0545-4c24-9f47-fd5bf5b6ee20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:27.949834 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6c05e-0545-4c24-9f47-fd5bf5b6ee20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:27.950109 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6c05e-0545-4c24-9f47-fd5bf5b6ee20 admin admin] PATH_INFO: `/v3/users/460b6517023c46578e9f0ff7e690351d` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:27.952229 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e6c05e-0545-4c24-9f47-fd5bf5b6ee20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=460b6517023c46578e9f0ff7e690351d)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:27.955003 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e6c05e-0545-4c24-9f47-fd5bf5b6ee20 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:28.003547 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-16e6c05e-0545-4c24-9f47-fd5bf5b6ee20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '460b6517023c46578e9f0ff7e690351d'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:28.006106 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412a2c9d-b71e-479d-a2d8-18521f11c22c None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:28.009812 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cc7b14-df83-4770-a30d-55fb268db172 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:28.010474 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cc7b14-df83-4770-a30d-55fb268db172 service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:28.010814 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cc7b14-df83-4770-a30d-55fb268db172 service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:28.011094 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cc7b14-df83-4770-a30d-55fb268db172 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:28.012978 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-e5cc7b14-df83-4770-a30d-55fb268db172 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:28.015623 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-16e6c05e-0545-4c24-9f47-fd5bf5b6ee20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '460b6517023c46578e9f0ff7e690351d'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:28.022894 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cc7b14-df83-4770-a30d-55fb268db172 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:28.023794 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-16e6c05e-0545-4c24-9f47-fd5bf5b6ee20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '460b6517023c46578e9f0ff7e690351d'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:28.024542 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cc7b14-df83-4770-a30d-55fb268db172 service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:28.033052 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-16e6c05e-0545-4c24-9f47-fd5bf5b6ee20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '460b6517023c46578e9f0ff7e690351d'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:28.033325 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1261/2492] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:02:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:02:28.033818 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-16e6c05e-0545-4c24-9f47-fd5bf5b6ee20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '460b6517023c46578e9f0ff7e690351d'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:28.040351 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1232/2493] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 02:02:27 2022] DELETE /identity/v3/users/460b6517023c46578e9f0ff7e690351d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:02:28.045262 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f06ffb2-f6ba-40d4-8ad4-ff3343a68ab3 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:28.045551 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f06ffb2-f6ba-40d4-8ad4-ff3343a68ab3 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:28.045827 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f06ffb2-f6ba-40d4-8ad4-ff3343a68ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:28.067559 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-7f06ffb2-f6ba-40d4-8ad4-ff3343a68ab3 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:02:28.068814 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-7f06ffb2-f6ba-40d4-8ad4-ff3343a68ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:28.131501 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1262/2494] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:02:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:02:28.141951 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e6c05e-0545-4c24-9f47-fd5bf5b6ee20 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:28.143485 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-16e6c05e-0545-4c24-9f47-fd5bf5b6ee20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:28.150244 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:02:28.212480 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e6c05e-0545-4c24-9f47-fd5bf5b6ee20 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:28.216332 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c3e17-db6b-4bec-b0c5-04b762c17372 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:28.216990 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c3e17-db6b-4bec-b0c5-04b762c17372 service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:28.217264 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c3e17-db6b-4bec-b0c5-04b762c17372 service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:28.217540 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c3e17-db6b-4bec-b0c5-04b762c17372 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:28.227082 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5c3e17-db6b-4bec-b0c5-04b762c17372 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:28.228579 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5c3e17-db6b-4bec-b0c5-04b762c17372 service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:28.238859 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1233/2495] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:02:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:02:28.440258 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f06ffb2-f6ba-40d4-8ad4-ff3343a68ab3 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:28.441744 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-7f06ffb2-f6ba-40d4-8ad4-ff3343a68ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:28.502416 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:02:28.534316 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f06ffb2-f6ba-40d4-8ad4-ff3343a68ab3 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:28.544141 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b7a4e-b165-41d5-9acb-7b1d184a17fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4e83df33b64a3ebf28f54e2d94fe63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:28.544764 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c3e17-db6b-4bec-b0c5-04b762c17372 service neutron] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:28.544997 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b7a4e-b165-41d5-9acb-7b1d184a17fa service nova] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:28.545447 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b7a4e-b165-41d5-9acb-7b1d184a17fa service nova] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:28.545721 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b7a4e-b165-41d5-9acb-7b1d184a17fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:28.547317 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-6f5c3e17-db6b-4bec-b0c5-04b762c17372 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:28.548274 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-a44b7a4e-b165-41d5-9acb-7b1d184a17fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:28.585052 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c3e17-db6b-4bec-b0c5-04b762c17372 service neutron] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:28.595851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98400ce-ed0f-49e1-9073-8dc401792141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:28.596746 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98400ce-ed0f-49e1-9073-8dc401792141 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:28.597033 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98400ce-ed0f-49e1-9073-8dc401792141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:28.597312 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98400ce-ed0f-49e1-9073-8dc401792141 admin admin] PATH_INFO: `/v3/projects/586e177d2f7341d985de3551a3e55697` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:28.598539 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98400ce-ed0f-49e1-9073-8dc401792141 admin admin] RBAC: Authorizing `identity:delete_project(project_id=586e177d2f7341d985de3551a3e55697)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:28.600528 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98400ce-ed0f-49e1-9073-8dc401792141 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:28.630985 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44b7a4e-b165-41d5-9acb-7b1d184a17fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:28.632921 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44b7a4e-b165-41d5-9acb-7b1d184a17fa service nova] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:28.644172 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1263/2496] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:02:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:02:28.644823 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b7a4e-b165-41d5-9acb-7b1d184a17fa service nova] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:28.646130 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-a44b7a4e-b165-41d5-9acb-7b1d184a17fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:28.657776 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-f98400ce-ed0f-49e1-9073-8dc401792141 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '586e177d2f7341d985de3551a3e55697', 'request_id': 'req-f98400ce-ed0f-49e1-9073-8dc401792141'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:28.666462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-f98400ce-ed0f-49e1-9073-8dc401792141 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '586e177d2f7341d985de3551a3e55697', 'request_id': 'req-f98400ce-ed0f-49e1-9073-8dc401792141'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:28.676803 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-f98400ce-ed0f-49e1-9073-8dc401792141 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '586e177d2f7341d985de3551a3e55697', 'request_id': 'req-f98400ce-ed0f-49e1-9073-8dc401792141'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:28.688491 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-f98400ce-ed0f-49e1-9073-8dc401792141 admin admin] The token cache is being invalidate because project 586e177d2f7341d985de3551a3e55697 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 02:02:28.688929 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-f98400ce-ed0f-49e1-9073-8dc401792141 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:28.689656 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-f98400ce-ed0f-49e1-9073-8dc401792141 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:28.691797 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1234/2497] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 02:02:28 2022] DELETE /identity/v3/projects/586e177d2f7341d985de3551a3e55697 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:02:28.696556 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fcbd92-b04f-4f79-98e5-4c43f9eea478 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:28.696845 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fcbd92-b04f-4f79-98e5-4c43f9eea478 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:28.697122 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fcbd92-b04f-4f79-98e5-4c43f9eea478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:28.718046 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-b1fcbd92-b04f-4f79-98e5-4c43f9eea478 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:02:28.719398 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-b1fcbd92-b04f-4f79-98e5-4c43f9eea478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:28.738130 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b7a4e-b165-41d5-9acb-7b1d184a17fa service nova] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:28.788180 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1235/2498] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:02:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 02:02:28.796254 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b4d0e-1fc0-4809-8735-f9829765ac9f None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:28.796852 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b4d0e-1fc0-4809-8735-f9829765ac9f None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:28.796852 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b4d0e-1fc0-4809-8735-f9829765ac9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:28.806430 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a3b6b9-da9c-4a96-bf9f-4c05b6e6c4c0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5bf3baee1b843fa89b65e5485f6bebb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:28.807174 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a3b6b9-da9c-4a96-bf9f-4c05b6e6c4c0 service placement] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:28.807562 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a3b6b9-da9c-4a96-bf9f-4c05b6e6c4c0 service placement] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:28.807889 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a3b6b9-da9c-4a96-bf9f-4c05b6e6c4c0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:28.810199 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-36a3b6b9-da9c-4a96-bf9f-4c05b6e6c4c0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:28.816056 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-002b4d0e-1fc0-4809-8735-f9829765ac9f None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:02:28.817291 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-002b4d0e-1fc0-4809-8735-f9829765ac9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:28.822678 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a3b6b9-da9c-4a96-bf9f-4c05b6e6c4c0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:28.823980 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a3b6b9-da9c-4a96-bf9f-4c05b6e6c4c0 service placement] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:28.828525 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1236/2499] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:02:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 02:02:28.829742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b4d0e-1fc0-4809-8735-f9829765ac9f None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:28.837395 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1264/2500] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:02:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:02:28.841920 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b4d0e-1fc0-4809-8735-f9829765ac9f None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:28.846920 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fdacbd-8e3f-416e-baa7-113798f80226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:28.847620 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fdacbd-8e3f-416e-baa7-113798f80226 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:28.847928 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fdacbd-8e3f-416e-baa7-113798f80226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:28.848200 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fdacbd-8e3f-416e-baa7-113798f80226 admin admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:28.848930 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fdacbd-8e3f-416e-baa7-113798f80226 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 02:02:28.849231 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fdacbd-8e3f-416e-baa7-113798f80226 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:28.850827 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fdacbd-8e3f-416e-baa7-113798f80226 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:28.864870 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1237/2501] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 02:02:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 02:02:28.869713 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a3b6b9-da9c-4a96-bf9f-4c05b6e6c4c0 service placement] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:28.882696 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a3b6b9-da9c-4a96-bf9f-4c05b6e6c4c0 service placement] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:28.887807 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14072ff-d66b-48b6-8f93-41ea4e823eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:28.888549 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14072ff-d66b-48b6-8f93-41ea4e823eee admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:28.888892 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14072ff-d66b-48b6-8f93-41ea4e823eee admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:28.889223 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14072ff-d66b-48b6-8f93-41ea4e823eee admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:28.890425 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14072ff-d66b-48b6-8f93-41ea4e823eee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:28.892051 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14072ff-d66b-48b6-8f93-41ea4e823eee admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:28.908923 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:02:28.927465 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1265/2502] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 02:02:28 2022] POST /identity/v3/projects => generated 361 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:02:28.930053 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fdacbd-8e3f-416e-baa7-113798f80226 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:28.943146 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fdacbd-8e3f-416e-baa7-113798f80226 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:28.963511 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:02:29.045026 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1968d3ba-e469-4a06-ab46-bd04dbfd9303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:29.045994 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968d3ba-e469-4a06-ab46-bd04dbfd9303 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:29.046419 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968d3ba-e469-4a06-ab46-bd04dbfd9303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:29.046845 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968d3ba-e469-4a06-ab46-bd04dbfd9303 admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:29.047895 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1968d3ba-e469-4a06-ab46-bd04dbfd9303 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:29.050257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1968d3ba-e469-4a06-ab46-bd04dbfd9303 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:29.077182 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1238/2503] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 02:02:28 2022] POST /identity/v3/users => generated 281 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:02:29.081357 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14072ff-d66b-48b6-8f93-41ea4e823eee admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:29.092558 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14072ff-d66b-48b6-8f93-41ea4e823eee admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:29.098923 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c8447f-23df-43e0-81e2-0be6a7cd8317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:29.099833 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8447f-23df-43e0-81e2-0be6a7cd8317 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:29.100140 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8447f-23df-43e0-81e2-0be6a7cd8317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:29.100467 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8447f-23df-43e0-81e2-0be6a7cd8317 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:29.101232 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c8447f-23df-43e0-81e2-0be6a7cd8317 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:29.103329 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c8447f-23df-43e0-81e2-0be6a7cd8317 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:29.123267 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1266/2504] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:02:29.125732 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1968d3ba-e469-4a06-ab46-bd04dbfd9303 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:29.136570 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1968d3ba-e469-4a06-ab46-bd04dbfd9303 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:29.143096 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b968ab69-2148-41df-ab1f-35da07371105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:29.143898 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b968ab69-2148-41df-ab1f-35da07371105 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:29.144170 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b968ab69-2148-41df-ab1f-35da07371105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:29.144464 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b968ab69-2148-41df-ab1f-35da07371105 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:29.145226 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b968ab69-2148-41df-ab1f-35da07371105 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:29.146869 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b968ab69-2148-41df-ab1f-35da07371105 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:29.164337 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1239/2505] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:02:29.165272 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c8447f-23df-43e0-81e2-0be6a7cd8317 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:29.181144 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c8447f-23df-43e0-81e2-0be6a7cd8317 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:29.188815 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba0edb-f4a8-4fc9-8c3d-303e3f12db71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:29.190573 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba0edb-f4a8-4fc9-8c3d-303e3f12db71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:29.190854 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba0edb-f4a8-4fc9-8c3d-303e3f12db71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:29.191133 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba0edb-f4a8-4fc9-8c3d-303e3f12db71 admin admin] PATH_INFO: `/v3/projects/c82e6027424347d2a8eca9317cd1498e/users/7faa5134b328498eac96df5431f92554/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:29.213497 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ba0edb-f4a8-4fc9-8c3d-303e3f12db71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c82e6027424347d2a8eca9317cd1498e, user_id=7faa5134b328498eac96df5431f92554, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:29.216264 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ba0edb-f4a8-4fc9-8c3d-303e3f12db71 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:29.235465 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1267/2506] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 02:02:29 2022] PUT /identity/v3/projects/c82e6027424347d2a8eca9317cd1498e/users/7faa5134b328498eac96df5431f92554/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:02:29.246350 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cfa720-d924-41f0-bf8a-3453985c186b None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:29.246698 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cfa720-d924-41f0-bf8a-3453985c186b None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:29.246919 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cfa720-d924-41f0-bf8a-3453985c186b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:29.277750 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-a2cfa720-d924-41f0-bf8a-3453985c186b None None] MFA Rules not processed for user `7faa5134b328498eac96df5431f92554`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:02:29.278821 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-a2cfa720-d924-41f0-bf8a-3453985c186b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:29.376700 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1240/2507] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:02:29 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:02:29.411729 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba0edb-f4a8-4fc9-8c3d-303e3f12db71 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:29.413528 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-39ba0edb-f4a8-4fc9-8c3d-303e3f12db71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:29.499136 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba0edb-f4a8-4fc9-8c3d-303e3f12db71 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:29.503119 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4cd11b-c2f2-49be-b902-ab368a718668 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:29.503991 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4cd11b-c2f2-49be-b902-ab368a718668 service cinder] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:29.504273 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4cd11b-c2f2-49be-b902-ab368a718668 service cinder] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:29.504729 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4cd11b-c2f2-49be-b902-ab368a718668 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:29.514996 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4cd11b-c2f2-49be-b902-ab368a718668 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:29.516844 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4cd11b-c2f2-49be-b902-ab368a718668 service cinder] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:29.541466 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1268/2508] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:02:29 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:02:29.558132 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cfa720-d924-41f0-bf8a-3453985c186b None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:29.645379 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cfa720-d924-41f0-bf8a-3453985c186b None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:29.649739 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f46b6-e304-415f-b87d-624199612d62 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:29.650261 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f46b6-e304-415f-b87d-624199612d62 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:29.650482 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f46b6-e304-415f-b87d-624199612d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:29.650685 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297e8c9b-67f3-421f-a8ea-583d34781155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:29.651461 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e8c9b-67f3-421f-a8ea-583d34781155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:29.651740 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e8c9b-67f3-421f-a8ea-583d34781155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:29.652010 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e8c9b-67f3-421f-a8ea-583d34781155 admin admin] PATH_INFO: `/v3/users/7faa5134b328498eac96df5431f92554` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:29.653234 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297e8c9b-67f3-421f-a8ea-583d34781155 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7faa5134b328498eac96df5431f92554)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:29.655188 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297e8c9b-67f3-421f-a8ea-583d34781155 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:29.671138 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-ad3f46b6-e304-415f-b87d-624199612d62 None None] MFA Rules not processed for user `168e6181ad22460990f1f7361b483364`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:02:29.672393 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-ad3f46b6-e304-415f-b87d-624199612d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:29.682609 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:02:29.720253 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-297e8c9b-67f3-421f-a8ea-583d34781155 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7faa5134b328498eac96df5431f92554'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:29.735502 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-297e8c9b-67f3-421f-a8ea-583d34781155 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7faa5134b328498eac96df5431f92554'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:29.741475 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1269/2509] 10.222.0.26 () {58 vars in 1009 bytes} [Thu Jun 23 02:02:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 02:02:29.744789 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-297e8c9b-67f3-421f-a8ea-583d34781155 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7faa5134b328498eac96df5431f92554'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:29.760093 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-297e8c9b-67f3-421f-a8ea-583d34781155 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7faa5134b328498eac96df5431f92554'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:29.760859 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-297e8c9b-67f3-421f-a8ea-583d34781155 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7faa5134b328498eac96df5431f92554'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:29.768293 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1241/2510] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 02:02:29 2022] DELETE /identity/v3/users/7faa5134b328498eac96df5431f92554 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:02:29.778955 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfca375-6daa-47ec-a327-ae39a8499d4d None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:29.779542 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfca375-6daa-47ec-a327-ae39a8499d4d None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:29.779829 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfca375-6daa-47ec-a327-ae39a8499d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:29.795822 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297e8c9b-67f3-421f-a8ea-583d34781155 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:29.797364 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-297e8c9b-67f3-421f-a8ea-583d34781155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:29.798532 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-7dfca375-6daa-47ec-a327-ae39a8499d4d None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:02:29.799820 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-7dfca375-6daa-47ec-a327-ae39a8499d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:29.863520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1270/2511] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:02:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:02:29.871540 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfca375-6daa-47ec-a327-ae39a8499d4d None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:29.872975 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-7dfca375-6daa-47ec-a327-ae39a8499d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:29.882345 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297e8c9b-67f3-421f-a8ea-583d34781155 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:29.885429 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:02:29.888507 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd17cae-a0e3-4175-9699-06ad8cb630f3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28e59688485241c4b6d51f99bbb3d059', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:29.889426 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd17cae-a0e3-4175-9699-06ad8cb630f3 service swift] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:29.889764 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd17cae-a0e3-4175-9699-06ad8cb630f3 service swift] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:29.890053 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd17cae-a0e3-4175-9699-06ad8cb630f3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:29.892482 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-dfd17cae-a0e3-4175-9699-06ad8cb630f3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:29.940083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfca375-6daa-47ec-a327-ae39a8499d4d None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:29.943828 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62e06f5-f3d3-47e7-9b21-52d9148933ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:29.944504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e06f5-f3d3-47e7-9b21-52d9148933ea service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:29.944796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e06f5-f3d3-47e7-9b21-52d9148933ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:29.945248 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e06f5-f3d3-47e7-9b21-52d9148933ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:29.956259 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62e06f5-f3d3-47e7-9b21-52d9148933ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:29.958702 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62e06f5-f3d3-47e7-9b21-52d9148933ea service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:29.958865 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd17cae-a0e3-4175-9699-06ad8cb630f3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:29.961109 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd17cae-a0e3-4175-9699-06ad8cb630f3 service swift] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:29.969826 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1271/2512] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:02:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 02:02:29.977472 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1242/2513] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Jun 23 02:02:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 23 02:02:30.485120 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62e06f5-f3d3-47e7-9b21-52d9148933ea service neutron] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:30.487971 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-c62e06f5-f3d3-47e7-9b21-52d9148933ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:30.546582 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62e06f5-f3d3-47e7-9b21-52d9148933ea service neutron] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:30.560372 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb084d0-97b2-4569-bc56-8ab271a4c598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:30.563426 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb084d0-97b2-4569-bc56-8ab271a4c598 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:30.563768 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb084d0-97b2-4569-bc56-8ab271a4c598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:30.564129 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb084d0-97b2-4569-bc56-8ab271a4c598 admin admin] PATH_INFO: `/v3/projects/c82e6027424347d2a8eca9317cd1498e` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:30.566952 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb084d0-97b2-4569-bc56-8ab271a4c598 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c82e6027424347d2a8eca9317cd1498e)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:30.570284 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb084d0-97b2-4569-bc56-8ab271a4c598 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:30.686909 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-cdb084d0-97b2-4569-bc56-8ab271a4c598 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c82e6027424347d2a8eca9317cd1498e', 'request_id': 'req-cdb084d0-97b2-4569-bc56-8ab271a4c598'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:30.697397 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-cdb084d0-97b2-4569-bc56-8ab271a4c598 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c82e6027424347d2a8eca9317cd1498e', 'request_id': 'req-cdb084d0-97b2-4569-bc56-8ab271a4c598'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:30.717948 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-cdb084d0-97b2-4569-bc56-8ab271a4c598 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c82e6027424347d2a8eca9317cd1498e', 'request_id': 'req-cdb084d0-97b2-4569-bc56-8ab271a4c598'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:30.750087 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-cdb084d0-97b2-4569-bc56-8ab271a4c598 admin admin] The token cache is being invalidate because project c82e6027424347d2a8eca9317cd1498e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 02:02:30.750392 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-cdb084d0-97b2-4569-bc56-8ab271a4c598 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:30.751120 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-cdb084d0-97b2-4569-bc56-8ab271a4c598 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:30.753368 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1272/2514] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 02:02:30 2022] DELETE /identity/v3/projects/c82e6027424347d2a8eca9317cd1498e => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:02:30.758389 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896fbaaa-3c1a-4abf-8660-4dcd8b8bfe57 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:30.758682 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896fbaaa-3c1a-4abf-8660-4dcd8b8bfe57 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:30.758962 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896fbaaa-3c1a-4abf-8660-4dcd8b8bfe57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:30.784202 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-896fbaaa-3c1a-4abf-8660-4dcd8b8bfe57 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:02:30.785780 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-896fbaaa-3c1a-4abf-8660-4dcd8b8bfe57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:30.841666 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:02:30.927517 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1243/2515] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:02:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:02:30.928109 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6932a239-7ed1-4b43-81f2-dbbd603ab54a None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:30.928421 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6932a239-7ed1-4b43-81f2-dbbd603ab54a None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:30.928705 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6932a239-7ed1-4b43-81f2-dbbd603ab54a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:30.982033 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-6932a239-7ed1-4b43-81f2-dbbd603ab54a None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:02:30.983373 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-6932a239-7ed1-4b43-81f2-dbbd603ab54a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:31.013308 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1273/2516] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:02:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:02:31.023664 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896fbaaa-3c1a-4abf-8660-4dcd8b8bfe57 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:31.047021 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896fbaaa-3c1a-4abf-8660-4dcd8b8bfe57 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:31.053276 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ba22fe-c9b7-4d06-bd2d-2b3b0f80a1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:31.054016 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ba22fe-c9b7-4d06-bd2d-2b3b0f80a1eb admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:31.055640 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ba22fe-c9b7-4d06-bd2d-2b3b0f80a1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:31.055939 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ba22fe-c9b7-4d06-bd2d-2b3b0f80a1eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:31.056826 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ba22fe-c9b7-4d06-bd2d-2b3b0f80a1eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 02:02:31.057141 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ba22fe-c9b7-4d06-bd2d-2b3b0f80a1eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:31.059478 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ba22fe-c9b7-4d06-bd2d-2b3b0f80a1eb admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:31.087755 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1244/2517] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 02:02:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 02:02:31.092092 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6932a239-7ed1-4b43-81f2-dbbd603ab54a None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:31.114595 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6932a239-7ed1-4b43-81f2-dbbd603ab54a None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:31.134168 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91189bd-a0c7-45d5-983c-b3fba9a381d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:31.135078 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91189bd-a0c7-45d5-983c-b3fba9a381d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:31.136327 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91189bd-a0c7-45d5-983c-b3fba9a381d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:31.137060 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91189bd-a0c7-45d5-983c-b3fba9a381d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:31.138413 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91189bd-a0c7-45d5-983c-b3fba9a381d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:31.145723 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91189bd-a0c7-45d5-983c-b3fba9a381d5 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:31.233843 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1274/2518] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 02:02:31 2022] POST /identity/v3/projects => generated 375 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:02:31.235405 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ba22fe-c9b7-4d06-bd2d-2b3b0f80a1eb admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:31.260379 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ba22fe-c9b7-4d06-bd2d-2b3b0f80a1eb admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:31.388616 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91189bd-a0c7-45d5-983c-b3fba9a381d5 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:31.392920 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-f91189bd-a0c7-45d5-983c-b3fba9a381d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:31.423964 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b32850-3e02-4ad3-a505-4bfe7f84e996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:31.427520 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b32850-3e02-4ad3-a505-4bfe7f84e996 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:31.427815 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b32850-3e02-4ad3-a505-4bfe7f84e996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:31.437397 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b32850-3e02-4ad3-a505-4bfe7f84e996 admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:31.441851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b32850-3e02-4ad3-a505-4bfe7f84e996 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:31.452756 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b32850-3e02-4ad3-a505-4bfe7f84e996 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:31.512134 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1245/2519] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 02:02:31 2022] POST /identity/v3/users => generated 288 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 02:02:31.515091 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b32850-3e02-4ad3-a505-4bfe7f84e996 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:31.521953 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-54b32850-3e02-4ad3-a505-4bfe7f84e996 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:31.610606 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:02:31.621910 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91189bd-a0c7-45d5-983c-b3fba9a381d5 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:31.639884 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1588bf1a-af80-47bd-b945-5fe1eae9e622 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4e83df33b64a3ebf28f54e2d94fe63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:31.640916 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588bf1a-af80-47bd-b945-5fe1eae9e622 service nova] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:31.641529 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588bf1a-af80-47bd-b945-5fe1eae9e622 service nova] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:31.642109 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588bf1a-af80-47bd-b945-5fe1eae9e622 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:31.644234 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-1588bf1a-af80-47bd-b945-5fe1eae9e622 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:31.664677 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1588bf1a-af80-47bd-b945-5fe1eae9e622 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:31.666512 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1588bf1a-af80-47bd-b945-5fe1eae9e622 service nova] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:31.679623 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b32850-3e02-4ad3-a505-4bfe7f84e996 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:31.691128 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9525467e-b232-4ae6-bc14-1500adea62d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:31.691840 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9525467e-b232-4ae6-bc14-1500adea62d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:31.692125 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9525467e-b232-4ae6-bc14-1500adea62d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:31.692402 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9525467e-b232-4ae6-bc14-1500adea62d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:31.694345 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1275/2520] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:02:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:02:31.696977 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1588bf1a-af80-47bd-b945-5fe1eae9e622 service nova] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:31.697461 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-9525467e-b232-4ae6-bc14-1500adea62d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:31.717856 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9525467e-b232-4ae6-bc14-1500adea62d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:31.721767 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9525467e-b232-4ae6-bc14-1500adea62d6 service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:31.729115 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1588bf1a-af80-47bd-b945-5fe1eae9e622 service nova] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:31.739795 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee65b8a-7e75-4423-bab3-94da85e94d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:31.748998 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee65b8a-7e75-4423-bab3-94da85e94d95 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:31.749360 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee65b8a-7e75-4423-bab3-94da85e94d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:31.749677 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee65b8a-7e75-4423-bab3-94da85e94d95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:31.750683 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee65b8a-7e75-4423-bab3-94da85e94d95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:31.752402 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee65b8a-7e75-4423-bab3-94da85e94d95 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:31.757874 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1246/2521] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:02:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:02:31.765299 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1276/2522] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:02:31 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:02:31.769399 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9525467e-b232-4ae6-bc14-1500adea62d6 service neutron] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:31.784448 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9525467e-b232-4ae6-bc14-1500adea62d6 service neutron] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:31.790309 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed8ba3-f8d8-4d8d-be6f-5a94666d88fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:31.791513 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed8ba3-f8d8-4d8d-be6f-5a94666d88fa admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:31.791969 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed8ba3-f8d8-4d8d-be6f-5a94666d88fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:31.792739 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed8ba3-f8d8-4d8d-be6f-5a94666d88fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:31.793664 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed8ba3-f8d8-4d8d-be6f-5a94666d88fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:31.797370 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed8ba3-f8d8-4d8d-be6f-5a94666d88fa admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:31.808560 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1247/2523] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:02:31 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 02:02:31.812624 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee65b8a-7e75-4423-bab3-94da85e94d95 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:31.830586 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee65b8a-7e75-4423-bab3-94da85e94d95 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:31.836743 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e784d1e-df9b-4196-a4ff-6c3d284b1220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:31.837345 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e784d1e-df9b-4196-a4ff-6c3d284b1220 admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:31.837748 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e784d1e-df9b-4196-a4ff-6c3d284b1220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:31.838134 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e784d1e-df9b-4196-a4ff-6c3d284b1220 admin admin] PATH_INFO: `/v3/projects/43be3154a6ce48b3a4608a3d1eb7b452/users/173f7af6aaf848368f00560104604b6d/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:31.857059 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e784d1e-df9b-4196-a4ff-6c3d284b1220 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43be3154a6ce48b3a4608a3d1eb7b452, user_id=173f7af6aaf848368f00560104604b6d, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:31.860177 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e784d1e-df9b-4196-a4ff-6c3d284b1220 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:31.892784 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1277/2524] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 02:02:31 2022] PUT /identity/v3/projects/43be3154a6ce48b3a4608a3d1eb7b452/users/173f7af6aaf848368f00560104604b6d/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:02:31.903002 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003c5dc-8830-486a-a5c8-cbab36f5f02a None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:31.903445 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003c5dc-8830-486a-a5c8-cbab36f5f02a None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:31.903823 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003c5dc-8830-486a-a5c8-cbab36f5f02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:31.934623 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-6003c5dc-8830-486a-a5c8-cbab36f5f02a None None] MFA Rules not processed for user `173f7af6aaf848368f00560104604b6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:02:31.936083 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-6003c5dc-8830-486a-a5c8-cbab36f5f02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:31.966017 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:02:32.051593 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1248/2525] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:02:31 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:02:32.064332 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e784d1e-df9b-4196-a4ff-6c3d284b1220 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:32.070279 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-1e784d1e-df9b-4196-a4ff-6c3d284b1220 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:32.231933 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e784d1e-df9b-4196-a4ff-6c3d284b1220 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:32.240459 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d7399-6438-4437-90a7-f68959093e35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:32.242431 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d7399-6438-4437-90a7-f68959093e35 service cinder] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:32.242697 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d7399-6438-4437-90a7-f68959093e35 service cinder] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:32.245137 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d7399-6438-4437-90a7-f68959093e35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:32.271577 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9d7399-6438-4437-90a7-f68959093e35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:32.275372 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9d7399-6438-4437-90a7-f68959093e35 service cinder] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:32.310640 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1278/2526] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:02:32 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:02:42.037544 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=15 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:02:42.287949 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=45,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:02:49.674811 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ccf04e-435a-4ada-9e7f-eb053d83a42c None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:49.675385 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ccf04e-435a-4ada-9e7f-eb053d83a42c None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:49.675488 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ccf04e-435a-4ada-9e7f-eb053d83a42c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:49.697068 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-a5ccf04e-435a-4ada-9e7f-eb053d83a42c None None] MFA Rules not processed for user `168e6181ad22460990f1f7361b483364`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:02:49.698424 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-a5ccf04e-435a-4ada-9e7f-eb053d83a42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:49.754566 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1249/2527] 10.222.0.26 () {58 vars in 1009 bytes} [Thu Jun 23 02:02:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:02:49.766344 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d7399-6438-4437-90a7-f68959093e35 service cinder] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:49.767872 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-be9d7399-6438-4437-90a7-f68959093e35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:49.842288 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d7399-6438-4437-90a7-f68959093e35 service cinder] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:49.847577 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19088ddc-3428-4e54-a161-ba94c0d6e4ca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28e59688485241c4b6d51f99bbb3d059', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:49.848249 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19088ddc-3428-4e54-a161-ba94c0d6e4ca service swift] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:49.848533 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19088ddc-3428-4e54-a161-ba94c0d6e4ca service swift] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:49.848809 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19088ddc-3428-4e54-a161-ba94c0d6e4ca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:49.857320 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19088ddc-3428-4e54-a161-ba94c0d6e4ca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:49.858924 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19088ddc-3428-4e54-a161-ba94c0d6e4ca service swift] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:49.867465 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1279/2528] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Jun 23 02:02:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 02:02:50.750495 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ccf04e-435a-4ada-9e7f-eb053d83a42c None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:50.864847 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ccf04e-435a-4ada-9e7f-eb053d83a42c None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:50.870510 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb6149-1d6c-4371-a0ce-4c113434948b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:50.871785 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb6149-1d6c-4371-a0ce-4c113434948b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:50.872286 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb6149-1d6c-4371-a0ce-4c113434948b admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:50.872790 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb6149-1d6c-4371-a0ce-4c113434948b admin admin] PATH_INFO: `/v3/users/173f7af6aaf848368f00560104604b6d` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:50.874621 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb6149-1d6c-4371-a0ce-4c113434948b admin admin] RBAC: Authorizing `identity:delete_user(user_id=173f7af6aaf848368f00560104604b6d)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:50.877331 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb6149-1d6c-4371-a0ce-4c113434948b admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:50.953898 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-03fb6149-1d6c-4371-a0ce-4c113434948b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '173f7af6aaf848368f00560104604b6d'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:50.972290 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-03fb6149-1d6c-4371-a0ce-4c113434948b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '173f7af6aaf848368f00560104604b6d'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:50.989176 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-03fb6149-1d6c-4371-a0ce-4c113434948b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '173f7af6aaf848368f00560104604b6d'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:51.016682 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-03fb6149-1d6c-4371-a0ce-4c113434948b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '173f7af6aaf848368f00560104604b6d'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:51.019193 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-03fb6149-1d6c-4371-a0ce-4c113434948b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '173f7af6aaf848368f00560104604b6d'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:51.031525 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1250/2529] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 02:02:50 2022] DELETE /identity/v3/users/173f7af6aaf848368f00560104604b6d => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:02:51.036590 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e33275-360f-4267-ac0c-a90f011a7a3d None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:51.038116 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e33275-360f-4267-ac0c-a90f011a7a3d None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:51.038757 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e33275-360f-4267-ac0c-a90f011a7a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:51.070411 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-b6e33275-360f-4267-ac0c-a90f011a7a3d None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:02:51.072842 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-b6e33275-360f-4267-ac0c-a90f011a7a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:51.173480 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1280/2530] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:02:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:02:51.179630 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb6149-1d6c-4371-a0ce-4c113434948b admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:51.181210 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-03fb6149-1d6c-4371-a0ce-4c113434948b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:51.228082 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:02:51.276123 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb6149-1d6c-4371-a0ce-4c113434948b admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:51.280034 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9659360-a786-4d94-8cc4-3addfc5dcd4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:51.280871 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9659360-a786-4d94-8cc4-3addfc5dcd4d service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:51.281153 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9659360-a786-4d94-8cc4-3addfc5dcd4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:51.281573 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9659360-a786-4d94-8cc4-3addfc5dcd4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:51.299456 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9659360-a786-4d94-8cc4-3addfc5dcd4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:51.301033 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9659360-a786-4d94-8cc4-3addfc5dcd4d service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:51.328736 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1251/2531] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:02:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 02:02:51.878026 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e33275-360f-4267-ac0c-a90f011a7a3d None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:51.879680 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-b6e33275-360f-4267-ac0c-a90f011a7a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:51.903248 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e33275-360f-4267-ac0c-a90f011a7a3d None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:51.915262 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2195b53-871e-4323-bacc-79613c412ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:51.918440 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2195b53-871e-4323-bacc-79613c412ba7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:51.920337 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2195b53-871e-4323-bacc-79613c412ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:51.920732 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2195b53-871e-4323-bacc-79613c412ba7 admin admin] PATH_INFO: `/v3/projects/43be3154a6ce48b3a4608a3d1eb7b452` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:51.924120 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2195b53-871e-4323-bacc-79613c412ba7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43be3154a6ce48b3a4608a3d1eb7b452)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:51.928328 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2195b53-871e-4323-bacc-79613c412ba7 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:52.019719 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-a2195b53-871e-4323-bacc-79613c412ba7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43be3154a6ce48b3a4608a3d1eb7b452', 'request_id': 'req-a2195b53-871e-4323-bacc-79613c412ba7'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:52.028180 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-a2195b53-871e-4323-bacc-79613c412ba7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43be3154a6ce48b3a4608a3d1eb7b452', 'request_id': 'req-a2195b53-871e-4323-bacc-79613c412ba7'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:52.037616 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-a2195b53-871e-4323-bacc-79613c412ba7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43be3154a6ce48b3a4608a3d1eb7b452', 'request_id': 'req-a2195b53-871e-4323-bacc-79613c412ba7'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:52.051976 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-a2195b53-871e-4323-bacc-79613c412ba7 admin admin] The token cache is being invalidate because project 43be3154a6ce48b3a4608a3d1eb7b452 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 02:02:52.052781 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-a2195b53-871e-4323-bacc-79613c412ba7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:52.053471 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-a2195b53-871e-4323-bacc-79613c412ba7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:52.059275 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1281/2532] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 02:02:51 2022] DELETE /identity/v3/projects/43be3154a6ce48b3a4608a3d1eb7b452 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:02:52.062251 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcef2dd-f2c8-4484-a2bb-b863a6c40a5d None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:52.062687 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcef2dd-f2c8-4484-a2bb-b863a6c40a5d None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:52.062840 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcef2dd-f2c8-4484-a2bb-b863a6c40a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:52.083404 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-5bcef2dd-f2c8-4484-a2bb-b863a6c40a5d None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:02:52.084928 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-5bcef2dd-f2c8-4484-a2bb-b863a6c40a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:52.192575 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47c998-1a24-4c3e-a155-f2e8d482663c None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:52.192887 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47c998-1a24-4c3e-a155-f2e8d482663c None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:52.193164 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47c998-1a24-4c3e-a155-f2e8d482663c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:52.195389 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1252/2533] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:02:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:02:52.217852 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-5a47c998-1a24-4c3e-a155-f2e8d482663c None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:02:52.219302 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-5a47c998-1a24-4c3e-a155-f2e8d482663c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:52.238923 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1282/2534] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:02:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:02:52.247140 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcef2dd-f2c8-4484-a2bb-b863a6c40a5d None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:52.264008 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcef2dd-f2c8-4484-a2bb-b863a6c40a5d None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:52.272357 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6220e21-6fa3-498b-8a94-663434e48dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:52.273672 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6220e21-6fa3-498b-8a94-663434e48dc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:52.274089 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6220e21-6fa3-498b-8a94-663434e48dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:52.275393 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6220e21-6fa3-498b-8a94-663434e48dc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:52.276622 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6220e21-6fa3-498b-8a94-663434e48dc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 02:02:52.276790 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6220e21-6fa3-498b-8a94-663434e48dc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:52.280452 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6220e21-6fa3-498b-8a94-663434e48dc5 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:52.295806 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1253/2535] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 02:02:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 02:02:52.299779 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a47c998-1a24-4c3e-a155-f2e8d482663c None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:52.312247 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a47c998-1a24-4c3e-a155-f2e8d482663c None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:52.319808 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b73e8f1-38bb-4919-b457-99efe6edaf28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:52.320522 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b73e8f1-38bb-4919-b457-99efe6edaf28 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:52.320810 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b73e8f1-38bb-4919-b457-99efe6edaf28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:52.321089 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b73e8f1-38bb-4919-b457-99efe6edaf28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:52.322027 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b73e8f1-38bb-4919-b457-99efe6edaf28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:52.324456 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b73e8f1-38bb-4919-b457-99efe6edaf28 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:52.327952 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:02:52.374209 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1283/2536] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 02:02:52 2022] POST /identity/v3/projects => generated 375 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:02:52.376422 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6220e21-6fa3-498b-8a94-663434e48dc5 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:52.390315 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6220e21-6fa3-498b-8a94-663434e48dc5 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:52.468996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae2e074-5ae7-481a-99d4-0a58110bb4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:52.469854 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae2e074-5ae7-481a-99d4-0a58110bb4a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:52.470153 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae2e074-5ae7-481a-99d4-0a58110bb4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:52.470435 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae2e074-5ae7-481a-99d4-0a58110bb4a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:52.471171 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae2e074-5ae7-481a-99d4-0a58110bb4a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:52.473520 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae2e074-5ae7-481a-99d4-0a58110bb4a9 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:52.505221 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1254/2537] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 02:02:52 2022] POST /identity/v3/users => generated 288 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 02:02:52.506577 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b73e8f1-38bb-4919-b457-99efe6edaf28 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:52.521851 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b73e8f1-38bb-4919-b457-99efe6edaf28 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:52.529728 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d55868a-1795-4d4c-959f-20b0a8989f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:52.530561 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d55868a-1795-4d4c-959f-20b0a8989f4a admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:52.530996 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d55868a-1795-4d4c-959f-20b0a8989f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:52.531399 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d55868a-1795-4d4c-959f-20b0a8989f4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:52.532264 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d55868a-1795-4d4c-959f-20b0a8989f4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:52.534025 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d55868a-1795-4d4c-959f-20b0a8989f4a admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:52.550634 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1284/2538] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:02:52.551459 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae2e074-5ae7-481a-99d4-0a58110bb4a9 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:52.563869 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae2e074-5ae7-481a-99d4-0a58110bb4a9 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:52.575412 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1a7064-f16b-4861-b31b-9d566f45be77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:52.576107 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a7064-f16b-4861-b31b-9d566f45be77 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:52.576388 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a7064-f16b-4861-b31b-9d566f45be77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:52.576661 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a7064-f16b-4861-b31b-9d566f45be77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:52.577409 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1a7064-f16b-4861-b31b-9d566f45be77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:52.579089 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1a7064-f16b-4861-b31b-9d566f45be77 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:52.595256 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1255/2539] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:02:52.597359 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d55868a-1795-4d4c-959f-20b0a8989f4a admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:52.613316 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d55868a-1795-4d4c-959f-20b0a8989f4a admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:52.619057 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebcf233-c44c-430d-8462-bd311fe92ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:52.620073 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebcf233-c44c-430d-8462-bd311fe92ba6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:52.620360 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebcf233-c44c-430d-8462-bd311fe92ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:52.620635 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebcf233-c44c-430d-8462-bd311fe92ba6 admin admin] PATH_INFO: `/v3/projects/725e623f92b440b29d808e730a87d6ff/users/1abfeaa81791449482554a19aa3bd5b3/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:52.646464 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebcf233-c44c-430d-8462-bd311fe92ba6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=725e623f92b440b29d808e730a87d6ff, user_id=1abfeaa81791449482554a19aa3bd5b3, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:52.651645 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebcf233-c44c-430d-8462-bd311fe92ba6 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:52.679311 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1285/2540] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 02:02:52 2022] PUT /identity/v3/projects/725e623f92b440b29d808e730a87d6ff/users/1abfeaa81791449482554a19aa3bd5b3/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:02:52.685226 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f416f3-d05d-4e87-bddd-cf71623918b9 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:52.685516 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f416f3-d05d-4e87-bddd-cf71623918b9 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:52.685799 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f416f3-d05d-4e87-bddd-cf71623918b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:52.719260 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-98f416f3-d05d-4e87-bddd-cf71623918b9 None None] MFA Rules not processed for user `1abfeaa81791449482554a19aa3bd5b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:02:52.720518 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-98f416f3-d05d-4e87-bddd-cf71623918b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:52.754215 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:02:52.849677 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1256/2541] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:02:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:02:52.858929 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebcf233-c44c-430d-8462-bd311fe92ba6 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:52.861027 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-5ebcf233-c44c-430d-8462-bd311fe92ba6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:52.947645 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebcf233-c44c-430d-8462-bd311fe92ba6 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:52.951774 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ddd11e-7a7e-4ca2-883f-3068cdc9b2cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:52.952660 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ddd11e-7a7e-4ca2-883f-3068cdc9b2cc service cinder] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:52.953158 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ddd11e-7a7e-4ca2-883f-3068cdc9b2cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:52.953640 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ddd11e-7a7e-4ca2-883f-3068cdc9b2cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:52.964756 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ddd11e-7a7e-4ca2-883f-3068cdc9b2cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:52.966350 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ddd11e-7a7e-4ca2-883f-3068cdc9b2cc service cinder] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:52.990714 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1286/2542] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:02:52 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:02:56.194614 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f416f3-d05d-4e87-bddd-cf71623918b9 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:56.264135 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f416f3-d05d-4e87-bddd-cf71623918b9 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:56.268574 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda5fe39-01f6-4722-89c2-4b83588ddc1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:56.269473 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda5fe39-01f6-4722-89c2-4b83588ddc1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:56.269748 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda5fe39-01f6-4722-89c2-4b83588ddc1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:56.270030 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda5fe39-01f6-4722-89c2-4b83588ddc1b admin admin] PATH_INFO: `/v3/users/1abfeaa81791449482554a19aa3bd5b3` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:56.271262 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda5fe39-01f6-4722-89c2-4b83588ddc1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1abfeaa81791449482554a19aa3bd5b3)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:56.273272 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda5fe39-01f6-4722-89c2-4b83588ddc1b admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:56.332538 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-dda5fe39-01f6-4722-89c2-4b83588ddc1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1abfeaa81791449482554a19aa3bd5b3'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:56.344936 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-dda5fe39-01f6-4722-89c2-4b83588ddc1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1abfeaa81791449482554a19aa3bd5b3'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:56.353491 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-dda5fe39-01f6-4722-89c2-4b83588ddc1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1abfeaa81791449482554a19aa3bd5b3'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:56.369980 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-dda5fe39-01f6-4722-89c2-4b83588ddc1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1abfeaa81791449482554a19aa3bd5b3'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:56.371108 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-dda5fe39-01f6-4722-89c2-4b83588ddc1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1abfeaa81791449482554a19aa3bd5b3'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:56.378930 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1257/2543] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 02:02:56 2022] DELETE /identity/v3/users/1abfeaa81791449482554a19aa3bd5b3 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:02:56.383828 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c25a6a-196e-40da-a09a-ec43f83237d5 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:56.384811 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c25a6a-196e-40da-a09a-ec43f83237d5 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:56.385093 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c25a6a-196e-40da-a09a-ec43f83237d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:56.408169 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-c0c25a6a-196e-40da-a09a-ec43f83237d5 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:02:56.409460 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-c0c25a6a-196e-40da-a09a-ec43f83237d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:56.473138 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1287/2544] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:02:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:02:56.482818 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda5fe39-01f6-4722-89c2-4b83588ddc1b admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:56.484269 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-dda5fe39-01f6-4722-89c2-4b83588ddc1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:56.541784 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:02:56.568430 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda5fe39-01f6-4722-89c2-4b83588ddc1b admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:56.572460 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d200581-4104-4cc5-9a25-137eafdec9ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:56.573509 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d200581-4104-4cc5-9a25-137eafdec9ce service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:56.573873 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d200581-4104-4cc5-9a25-137eafdec9ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:56.574226 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d200581-4104-4cc5-9a25-137eafdec9ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:56.590069 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d200581-4104-4cc5-9a25-137eafdec9ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:56.591602 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d200581-4104-4cc5-9a25-137eafdec9ce service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:56.621398 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1258/2545] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:02:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:02:57.271252 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c25a6a-196e-40da-a09a-ec43f83237d5 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:57.272950 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-c0c25a6a-196e-40da-a09a-ec43f83237d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:57.286977 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c25a6a-196e-40da-a09a-ec43f83237d5 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:57.301632 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f360440b-9c92-490e-af9f-dbc7cb010f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:57.302923 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360440b-9c92-490e-af9f-dbc7cb010f2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:57.303498 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360440b-9c92-490e-af9f-dbc7cb010f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:57.303982 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360440b-9c92-490e-af9f-dbc7cb010f2d admin admin] PATH_INFO: `/v3/projects/725e623f92b440b29d808e730a87d6ff` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:57.305317 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f360440b-9c92-490e-af9f-dbc7cb010f2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=725e623f92b440b29d808e730a87d6ff)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:57.307405 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f360440b-9c92-490e-af9f-dbc7cb010f2d admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:57.358955 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:02:57.366874 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-f360440b-9c92-490e-af9f-dbc7cb010f2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '725e623f92b440b29d808e730a87d6ff', 'request_id': 'req-f360440b-9c92-490e-af9f-dbc7cb010f2d'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:57.374585 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-f360440b-9c92-490e-af9f-dbc7cb010f2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '725e623f92b440b29d808e730a87d6ff', 'request_id': 'req-f360440b-9c92-490e-af9f-dbc7cb010f2d'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:57.383089 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-f360440b-9c92-490e-af9f-dbc7cb010f2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '725e623f92b440b29d808e730a87d6ff', 'request_id': 'req-f360440b-9c92-490e-af9f-dbc7cb010f2d'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:57.394609 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-f360440b-9c92-490e-af9f-dbc7cb010f2d admin admin] The token cache is being invalidate because project 725e623f92b440b29d808e730a87d6ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 02:02:57.395357 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-f360440b-9c92-490e-af9f-dbc7cb010f2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:57.395554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-f360440b-9c92-490e-af9f-dbc7cb010f2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:02:57.397317 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1288/2546] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 02:02:57 2022] DELETE /identity/v3/projects/725e623f92b440b29d808e730a87d6ff => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:02:57.402352 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b10226-21c7-4405-8a16-14008bb2c52a None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:57.402784 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b10226-21c7-4405-8a16-14008bb2c52a None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:57.402908 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b10226-21c7-4405-8a16-14008bb2c52a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:57.421466 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-c7b10226-21c7-4405-8a16-14008bb2c52a None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:02:57.422714 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-c7b10226-21c7-4405-8a16-14008bb2c52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:57.484550 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1259/2547] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:02:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 02:02:57.492755 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4852574-23ea-43aa-9e59-3801cb9c1e73 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:57.493042 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4852574-23ea-43aa-9e59-3801cb9c1e73 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:57.493345 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4852574-23ea-43aa-9e59-3801cb9c1e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:57.520127 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-a4852574-23ea-43aa-9e59-3801cb9c1e73 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:02:57.522565 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-a4852574-23ea-43aa-9e59-3801cb9c1e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:57.545269 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1289/2548] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:02:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:02:57.546016 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b10226-21c7-4405-8a16-14008bb2c52a None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:57.556853 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b10226-21c7-4405-8a16-14008bb2c52a None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:57.564341 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55febf-61ae-4e14-b5a0-abdf7d7e64d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:57.565034 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55febf-61ae-4e14-b5a0-abdf7d7e64d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:57.565364 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55febf-61ae-4e14-b5a0-abdf7d7e64d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:57.566151 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55febf-61ae-4e14-b5a0-abdf7d7e64d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:57.568208 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d55febf-61ae-4e14-b5a0-abdf7d7e64d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 02:02:57.568561 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d55febf-61ae-4e14-b5a0-abdf7d7e64d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:57.570725 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d55febf-61ae-4e14-b5a0-abdf7d7e64d3 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:57.593688 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1260/2549] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 02:02:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 02:02:57.598617 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4852574-23ea-43aa-9e59-3801cb9c1e73 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:57.614088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4852574-23ea-43aa-9e59-3801cb9c1e73 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:57.621087 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275e18be-b3a5-4a3f-859d-82750861818d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:57.621761 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e18be-b3a5-4a3f-859d-82750861818d admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:57.622037 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e18be-b3a5-4a3f-859d-82750861818d admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:57.622305 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e18be-b3a5-4a3f-859d-82750861818d admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:57.623033 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275e18be-b3a5-4a3f-859d-82750861818d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:57.624988 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275e18be-b3a5-4a3f-859d-82750861818d admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:57.721037 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1290/2550] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 02:02:57 2022] POST /identity/v3/projects => generated 371 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:02:57.725418 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55febf-61ae-4e14-b5a0-abdf7d7e64d3 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:57.736284 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55febf-61ae-4e14-b5a0-abdf7d7e64d3 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:57.795139 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d457c-9f24-4a66-ada4-14108f8adfee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:57.795848 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d457c-9f24-4a66-ada4-14108f8adfee admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:57.796126 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d457c-9f24-4a66-ada4-14108f8adfee admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:57.796399 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d457c-9f24-4a66-ada4-14108f8adfee admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:57.797088 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d457c-9f24-4a66-ada4-14108f8adfee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:57.799006 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d457c-9f24-4a66-ada4-14108f8adfee admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:57.824128 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1261/2551] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 02:02:57 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:02:57.828107 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275e18be-b3a5-4a3f-859d-82750861818d admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:57.838616 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275e18be-b3a5-4a3f-859d-82750861818d admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:57.843696 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c8b3e-2dfb-424a-bf54-4c7e84bc8a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:57.844355 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c8b3e-2dfb-424a-bf54-4c7e84bc8a23 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:57.844635 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c8b3e-2dfb-424a-bf54-4c7e84bc8a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:57.845013 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c8b3e-2dfb-424a-bf54-4c7e84bc8a23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:57.845641 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1c8b3e-2dfb-424a-bf54-4c7e84bc8a23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:57.847302 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1c8b3e-2dfb-424a-bf54-4c7e84bc8a23 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:57.858275 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1291/2552] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:02:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:02:57.862491 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d457c-9f24-4a66-ada4-14108f8adfee admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:57.873362 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d457c-9f24-4a66-ada4-14108f8adfee admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:57.878997 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc20ba97-fe80-4d17-96a0-9927e09ff0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:57.879701 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc20ba97-fe80-4d17-96a0-9927e09ff0e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:57.879970 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc20ba97-fe80-4d17-96a0-9927e09ff0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:57.880241 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc20ba97-fe80-4d17-96a0-9927e09ff0e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:57.881007 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc20ba97-fe80-4d17-96a0-9927e09ff0e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:57.882647 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc20ba97-fe80-4d17-96a0-9927e09ff0e3 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:57.892629 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1262/2553] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:02:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:02:57.897037 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c8b3e-2dfb-424a-bf54-4c7e84bc8a23 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:57.913079 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c8b3e-2dfb-424a-bf54-4c7e84bc8a23 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:57.919073 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449ada29-fb12-42c6-bba6-cf0098a804eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:57.920006 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ada29-fb12-42c6-bba6-cf0098a804eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:57.920287 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ada29-fb12-42c6-bba6-cf0098a804eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:57.920563 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ada29-fb12-42c6-bba6-cf0098a804eb admin admin] PATH_INFO: `/v3/projects/de5fab60406c42d3be1c783e6e774bf1/users/85d3bb3d6b7d4a45b6dffae1e346b406/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:57.936918 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449ada29-fb12-42c6-bba6-cf0098a804eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=de5fab60406c42d3be1c783e6e774bf1, user_id=85d3bb3d6b7d4a45b6dffae1e346b406, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:57.939644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449ada29-fb12-42c6-bba6-cf0098a804eb admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:57.955114 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1292/2554] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 02:02:57 2022] PUT /identity/v3/projects/de5fab60406c42d3be1c783e6e774bf1/users/85d3bb3d6b7d4a45b6dffae1e346b406/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:02:57.967054 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc23bc88-3cd8-4934-9516-d3d4e4e2ed09 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:57.967418 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc23bc88-3cd8-4934-9516-d3d4e4e2ed09 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:57.967679 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc23bc88-3cd8-4934-9516-d3d4e4e2ed09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:57.996775 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-fc23bc88-3cd8-4934-9516-d3d4e4e2ed09 None None] MFA Rules not processed for user `85d3bb3d6b7d4a45b6dffae1e346b406`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:02:57.997931 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-fc23bc88-3cd8-4934-9516-d3d4e4e2ed09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:58.040767 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:02:58.066350 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1263/2555] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:02:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:02:58.078221 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449ada29-fb12-42c6-bba6-cf0098a804eb admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:02:58.080260 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-449ada29-fb12-42c6-bba6-cf0098a804eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:02:58.143524 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449ada29-fb12-42c6-bba6-cf0098a804eb admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:02:58.148011 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ad308-69ec-4658-a36e-10507f1d0cc6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c081b4cf34146ca8eb197b62a133433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:02:58.148869 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ad308-69ec-4658-a36e-10507f1d0cc6 service glance] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:02:58.149164 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ad308-69ec-4658-a36e-10507f1d0cc6 service glance] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:02:58.149535 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ad308-69ec-4658-a36e-10507f1d0cc6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:02:58.161118 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ad308-69ec-4658-a36e-10507f1d0cc6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:02:58.163046 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ad308-69ec-4658-a36e-10507f1d0cc6 service glance] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:02:58.175094 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1293/2556] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:02:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:03:07.045562 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7246af2-4e4f-4099-8732-434b13607cef None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:07.046229 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7246af2-4e4f-4099-8732-434b13607cef None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:07.046229 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7246af2-4e4f-4099-8732-434b13607cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:07.067481 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-c7246af2-4e4f-4099-8732-434b13607cef None None] MFA Rules not processed for user `168e6181ad22460990f1f7361b483364`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:03:07.068931 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-c7246af2-4e4f-4099-8732-434b13607cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:03:07.133349 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1264/2557] 10.222.0.26 () {58 vars in 967 bytes} [Thu Jun 23 02:03:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:03:07.147817 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ad308-69ec-4658-a36e-10507f1d0cc6 service glance] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:07.150212 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-7a5ad308-69ec-4658-a36e-10507f1d0cc6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:03:07.237350 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ad308-69ec-4658-a36e-10507f1d0cc6 service glance] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:07.242378 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79970790-d1e4-4bfc-8142-c45295738547 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28e59688485241c4b6d51f99bbb3d059', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:07.243037 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79970790-d1e4-4bfc-8142-c45295738547 service swift] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:07.243358 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79970790-d1e4-4bfc-8142-c45295738547 service swift] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:07.243682 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79970790-d1e4-4bfc-8142-c45295738547 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:07.253394 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79970790-d1e4-4bfc-8142-c45295738547 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:07.255744 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79970790-d1e4-4bfc-8142-c45295738547 service swift] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:07.259524 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:03:07.275081 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1294/2558] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Jun 23 02:03:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 02:03:12.599243 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7246af2-4e4f-4099-8732-434b13607cef None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:12.600725 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-c7246af2-4e4f-4099-8732-434b13607cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:03:12.658916 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7246af2-4e4f-4099-8732-434b13607cef None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:12.662410 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8732ec84-9d61-4050-8df9-4d2624010bf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:12.663062 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8732ec84-9d61-4050-8df9-4d2624010bf8 service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:12.663390 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8732ec84-9d61-4050-8df9-4d2624010bf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:12.663667 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8732ec84-9d61-4050-8df9-4d2624010bf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:12.665444 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-8732ec84-9d61-4050-8df9-4d2624010bf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:03:12.728930 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8732ec84-9d61-4050-8df9-4d2624010bf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:12.730362 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8732ec84-9d61-4050-8df9-4d2624010bf8 service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:12.740278 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1265/2559] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:03:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:03:15.944324 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79970790-d1e4-4bfc-8142-c45295738547 service swift] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:15.946016 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-79970790-d1e4-4bfc-8142-c45295738547 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:03:15.963009 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79970790-d1e4-4bfc-8142-c45295738547 service swift] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:15.968320 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b90b1f-9d33-44ac-9356-2bdba5f630dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:15.969504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b90b1f-9d33-44ac-9356-2bdba5f630dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:15.969810 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b90b1f-9d33-44ac-9356-2bdba5f630dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:15.970085 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b90b1f-9d33-44ac-9356-2bdba5f630dc admin admin] PATH_INFO: `/v3/users/ababea3e976f46889f2b1980edfee883` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:15.971452 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b90b1f-9d33-44ac-9356-2bdba5f630dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ababea3e976f46889f2b1980edfee883)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:15.973429 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b90b1f-9d33-44ac-9356-2bdba5f630dc admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:16.039680 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-79b90b1f-9d33-44ac-9356-2bdba5f630dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ababea3e976f46889f2b1980edfee883'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:16.056532 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-79b90b1f-9d33-44ac-9356-2bdba5f630dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ababea3e976f46889f2b1980edfee883'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:16.065637 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-79b90b1f-9d33-44ac-9356-2bdba5f630dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ababea3e976f46889f2b1980edfee883'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:16.073362 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-79b90b1f-9d33-44ac-9356-2bdba5f630dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ababea3e976f46889f2b1980edfee883'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:16.074142 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-79b90b1f-9d33-44ac-9356-2bdba5f630dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ababea3e976f46889f2b1980edfee883'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:16.081170 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1295/2560] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 02:03:15 2022] DELETE /identity/v3/users/ababea3e976f46889f2b1980edfee883 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:03:16.282266 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8732ec84-9d61-4050-8df9-4d2624010bf8 service neutron] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:16.294887 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8732ec84-9d61-4050-8df9-4d2624010bf8 service neutron] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:16.334150 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=100 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:03:16.359248 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa80dc5-c66b-4612-b0b7-a523ff2d218a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:16.360089 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa80dc5-c66b-4612-b0b7-a523ff2d218a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:16.360369 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa80dc5-c66b-4612-b0b7-a523ff2d218a admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:16.360646 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa80dc5-c66b-4612-b0b7-a523ff2d218a admin admin] PATH_INFO: `/v3/projects/96afcc3674a54ffb86c1f0298a527e61` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:16.361854 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa80dc5-c66b-4612-b0b7-a523ff2d218a admin admin] RBAC: Authorizing `identity:delete_project(project_id=96afcc3674a54ffb86c1f0298a527e61)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:16.363894 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa80dc5-c66b-4612-b0b7-a523ff2d218a admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:16.410176 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-afa80dc5-c66b-4612-b0b7-a523ff2d218a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96afcc3674a54ffb86c1f0298a527e61', 'request_id': 'req-afa80dc5-c66b-4612-b0b7-a523ff2d218a'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:16.416384 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-afa80dc5-c66b-4612-b0b7-a523ff2d218a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96afcc3674a54ffb86c1f0298a527e61', 'request_id': 'req-afa80dc5-c66b-4612-b0b7-a523ff2d218a'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:16.425907 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-afa80dc5-c66b-4612-b0b7-a523ff2d218a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96afcc3674a54ffb86c1f0298a527e61', 'request_id': 'req-afa80dc5-c66b-4612-b0b7-a523ff2d218a'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:16.436815 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-afa80dc5-c66b-4612-b0b7-a523ff2d218a admin admin] The token cache is being invalidate because project 96afcc3674a54ffb86c1f0298a527e61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 02:03:16.437101 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-afa80dc5-c66b-4612-b0b7-a523ff2d218a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:16.437799 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-afa80dc5-c66b-4612-b0b7-a523ff2d218a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:16.439511 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1266/2561] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 02:03:16 2022] DELETE /identity/v3/projects/96afcc3674a54ffb86c1f0298a527e61 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:03:16.444764 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57272114-c983-4d60-8a0a-dc0f58d7fc01 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:16.445053 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57272114-c983-4d60-8a0a-dc0f58d7fc01 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:16.445324 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57272114-c983-4d60-8a0a-dc0f58d7fc01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:16.464075 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-57272114-c983-4d60-8a0a-dc0f58d7fc01 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:03:16.465302 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-57272114-c983-4d60-8a0a-dc0f58d7fc01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:03:16.528586 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1296/2562] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:03:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:03:16.538943 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469a79b8-5947-4cbb-8fde-e45649a2a161 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:16.539256 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469a79b8-5947-4cbb-8fde-e45649a2a161 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:16.539535 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469a79b8-5947-4cbb-8fde-e45649a2a161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:16.560456 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-469a79b8-5947-4cbb-8fde-e45649a2a161 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:03:16.561668 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-469a79b8-5947-4cbb-8fde-e45649a2a161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:03:16.570735 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1267/2563] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:03:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:03:16.575444 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57272114-c983-4d60-8a0a-dc0f58d7fc01 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:16.586008 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57272114-c983-4d60-8a0a-dc0f58d7fc01 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:16.592104 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e50cf0e-fc0b-443f-8185-d599cc9d3efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:16.592745 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50cf0e-fc0b-443f-8185-d599cc9d3efe admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:16.593022 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50cf0e-fc0b-443f-8185-d599cc9d3efe admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:16.593299 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50cf0e-fc0b-443f-8185-d599cc9d3efe admin admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:16.594055 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e50cf0e-fc0b-443f-8185-d599cc9d3efe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 02:03:16.594367 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e50cf0e-fc0b-443f-8185-d599cc9d3efe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:16.596092 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e50cf0e-fc0b-443f-8185-d599cc9d3efe admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:16.609010 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1297/2564] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 02:03:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 02:03:16.616128 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469a79b8-5947-4cbb-8fde-e45649a2a161 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:16.627243 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469a79b8-5947-4cbb-8fde-e45649a2a161 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:16.632991 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc79481-3e4b-4388-9055-916b703a638a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:16.633650 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc79481-3e4b-4388-9055-916b703a638a admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:16.633956 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc79481-3e4b-4388-9055-916b703a638a admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:16.634189 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc79481-3e4b-4388-9055-916b703a638a admin admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:16.634893 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc79481-3e4b-4388-9055-916b703a638a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:16.636846 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc79481-3e4b-4388-9055-916b703a638a admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:16.664887 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1268/2565] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 02:03:16 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:03:16.669640 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e50cf0e-fc0b-443f-8185-d599cc9d3efe admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:16.681632 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e50cf0e-fc0b-443f-8185-d599cc9d3efe admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:16.730698 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:03:16.739470 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a57a7a-9374-48e5-a589-7d2b63eb4ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:16.740307 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a57a7a-9374-48e5-a589-7d2b63eb4ac1 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:16.740774 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a57a7a-9374-48e5-a589-7d2b63eb4ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:16.741230 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a57a7a-9374-48e5-a589-7d2b63eb4ac1 admin admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:16.741920 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a57a7a-9374-48e5-a589-7d2b63eb4ac1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:16.744438 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a57a7a-9374-48e5-a589-7d2b63eb4ac1 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:16.764443 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1298/2566] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 02:03:16 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 02:03:16.768665 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc79481-3e4b-4388-9055-916b703a638a admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:16.778735 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc79481-3e4b-4388-9055-916b703a638a admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:16.783308 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a195e78-2c2d-4519-aed4-47aab29d3a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:16.783955 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a195e78-2c2d-4519-aed4-47aab29d3a06 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:16.784230 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a195e78-2c2d-4519-aed4-47aab29d3a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:16.784497 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a195e78-2c2d-4519-aed4-47aab29d3a06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:16.785225 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a195e78-2c2d-4519-aed4-47aab29d3a06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:16.786805 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a195e78-2c2d-4519-aed4-47aab29d3a06 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:16.797676 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1269/2567] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:03:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:03:16.802246 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a57a7a-9374-48e5-a589-7d2b63eb4ac1 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:16.816818 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a57a7a-9374-48e5-a589-7d2b63eb4ac1 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:16.825041 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96b07f-f049-4340-825c-6fb982dfae41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:16.825929 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96b07f-f049-4340-825c-6fb982dfae41 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:16.826349 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96b07f-f049-4340-825c-6fb982dfae41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:16.826656 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96b07f-f049-4340-825c-6fb982dfae41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:16.827743 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d96b07f-f049-4340-825c-6fb982dfae41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:16.829740 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d96b07f-f049-4340-825c-6fb982dfae41 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:16.841046 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1299/2568] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:03:16 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:03:16.845760 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a195e78-2c2d-4519-aed4-47aab29d3a06 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:16.857444 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a195e78-2c2d-4519-aed4-47aab29d3a06 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:16.864510 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25af231-3298-486e-ace7-858667f1fd0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:16.865672 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25af231-3298-486e-ace7-858667f1fd0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:16.865950 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25af231-3298-486e-ace7-858667f1fd0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:16.866276 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25af231-3298-486e-ace7-858667f1fd0a admin admin] PATH_INFO: `/v3/projects/07da57919f294173b2dc00509c4a6e0f/users/ffe72dd809a84bd5aae15a85f081324e/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:16.883696 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25af231-3298-486e-ace7-858667f1fd0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=07da57919f294173b2dc00509c4a6e0f, user_id=ffe72dd809a84bd5aae15a85f081324e, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:16.886388 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25af231-3298-486e-ace7-858667f1fd0a admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:16.901347 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1270/2569] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 02:03:16 2022] PUT /identity/v3/projects/07da57919f294173b2dc00509c4a6e0f/users/ffe72dd809a84bd5aae15a85f081324e/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:03:16.906548 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c584f300-3e89-4ffd-bcc5-9d9a92829d6c None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:16.906867 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c584f300-3e89-4ffd-bcc5-9d9a92829d6c None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:16.907147 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c584f300-3e89-4ffd-bcc5-9d9a92829d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:16.925118 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-c584f300-3e89-4ffd-bcc5-9d9a92829d6c None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:03:16.926366 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-c584f300-3e89-4ffd-bcc5-9d9a92829d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:03:16.990658 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1300/2570] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:03:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:03:17.000421 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25af231-3298-486e-ace7-858667f1fd0a admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:17.002214 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-b25af231-3298-486e-ace7-858667f1fd0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:03:17.060949 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25af231-3298-486e-ace7-858667f1fd0a admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:17.065141 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba3153d-fad0-45cf-8958-b88aed4c3cdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:17.065795 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3153d-fad0-45cf-8958-b88aed4c3cdb service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:17.066078 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3153d-fad0-45cf-8958-b88aed4c3cdb service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:17.066357 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3153d-fad0-45cf-8958-b88aed4c3cdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:17.076472 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba3153d-fad0-45cf-8958-b88aed4c3cdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:17.077921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba3153d-fad0-45cf-8958-b88aed4c3cdb service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:17.088187 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1271/2571] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:03:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:03:21.105871 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aded130-8d2a-4314-9b19-59d928448b1c None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:21.106389 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aded130-8d2a-4314-9b19-59d928448b1c None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:21.106473 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aded130-8d2a-4314-9b19-59d928448b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:21.136343 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-1aded130-8d2a-4314-9b19-59d928448b1c None None] MFA Rules not processed for user `ffe72dd809a84bd5aae15a85f081324e`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:03:21.137564 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-1aded130-8d2a-4314-9b19-59d928448b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:03:21.210360 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1301/2572] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:03:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:03:21.226143 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba3153d-fad0-45cf-8958-b88aed4c3cdb service neutron] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:21.227739 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-aba3153d-fad0-45cf-8958-b88aed4c3cdb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:03:21.234389 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:03:21.312371 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba3153d-fad0-45cf-8958-b88aed4c3cdb service neutron] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:21.317913 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355e0e70-c5d4-46d1-abb0-82f9452be716 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4e83df33b64a3ebf28f54e2d94fe63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:21.318592 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e0e70-c5d4-46d1-abb0-82f9452be716 service nova] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:21.318869 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e0e70-c5d4-46d1-abb0-82f9452be716 service nova] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:21.319146 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e0e70-c5d4-46d1-abb0-82f9452be716 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:21.327976 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355e0e70-c5d4-46d1-abb0-82f9452be716 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:21.329737 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355e0e70-c5d4-46d1-abb0-82f9452be716 service nova] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:21.339704 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1272/2573] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:03:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:03:22.603123 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6c609-78a7-42b8-b26f-f48f599465fb None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:22.603737 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6c609-78a7-42b8-b26f-f48f599465fb None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:22.603737 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6c609-78a7-42b8-b26f-f48f599465fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:22.622244 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-4ae6c609-78a7-42b8-b26f-f48f599465fb None None] MFA Rules not processed for user `ffe72dd809a84bd5aae15a85f081324e`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:03:22.623592 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-4ae6c609-78a7-42b8-b26f-f48f599465fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:03:22.631104 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1302/2574] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:03:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:03:24.132874 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355e0e70-c5d4-46d1-abb0-82f9452be716 service nova] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:24.134365 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-355e0e70-c5d4-46d1-abb0-82f9452be716 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:03:24.150302 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355e0e70-c5d4-46d1-abb0-82f9452be716 service nova] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:24.155691 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8977303d-7fae-42ee-9ac0-03d47ccc60f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:24.156649 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977303d-7fae-42ee-9ac0-03d47ccc60f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:24.157091 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977303d-7fae-42ee-9ac0-03d47ccc60f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:24.157376 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977303d-7fae-42ee-9ac0-03d47ccc60f1 admin admin] PATH_INFO: `/v3/users/85d3bb3d6b7d4a45b6dffae1e346b406` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:24.158610 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8977303d-7fae-42ee-9ac0-03d47ccc60f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85d3bb3d6b7d4a45b6dffae1e346b406)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:24.160582 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8977303d-7fae-42ee-9ac0-03d47ccc60f1 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:24.212616 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-8977303d-7fae-42ee-9ac0-03d47ccc60f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85d3bb3d6b7d4a45b6dffae1e346b406'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:24.226406 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-8977303d-7fae-42ee-9ac0-03d47ccc60f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85d3bb3d6b7d4a45b6dffae1e346b406'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:24.237996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-8977303d-7fae-42ee-9ac0-03d47ccc60f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85d3bb3d6b7d4a45b6dffae1e346b406'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:24.245831 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-8977303d-7fae-42ee-9ac0-03d47ccc60f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85d3bb3d6b7d4a45b6dffae1e346b406'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:24.246580 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-8977303d-7fae-42ee-9ac0-03d47ccc60f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85d3bb3d6b7d4a45b6dffae1e346b406'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:24.253619 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1273/2575] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 02:03:24 2022] DELETE /identity/v3/users/85d3bb3d6b7d4a45b6dffae1e346b406 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:03:24.258294 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db75a4a-b009-4666-95d6-d9067f3dd3fa None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:24.258803 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db75a4a-b009-4666-95d6-d9067f3dd3fa None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:24.258923 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db75a4a-b009-4666-95d6-d9067f3dd3fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:24.281784 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-8db75a4a-b009-4666-95d6-d9067f3dd3fa None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:03:24.283094 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-8db75a4a-b009-4666-95d6-d9067f3dd3fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:03:24.371595 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:03:24.395014 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8977303d-7fae-42ee-9ac0-03d47ccc60f1 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:24.396807 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-8977303d-7fae-42ee-9ac0-03d47ccc60f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:03:24.402894 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1303/2576] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:03:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:03:24.480793 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8977303d-7fae-42ee-9ac0-03d47ccc60f1 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:24.485578 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5815eef1-c41a-4e07-b0f7-37d214ac2f9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:24.487998 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5815eef1-c41a-4e07-b0f7-37d214ac2f9c service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:24.488651 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5815eef1-c41a-4e07-b0f7-37d214ac2f9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:24.489129 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5815eef1-c41a-4e07-b0f7-37d214ac2f9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:24.500733 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5815eef1-c41a-4e07-b0f7-37d214ac2f9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:24.502191 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5815eef1-c41a-4e07-b0f7-37d214ac2f9c service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:24.516868 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1274/2577] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:03:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 02:03:24.979042 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db75a4a-b009-4666-95d6-d9067f3dd3fa None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:24.980645 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-8db75a4a-b009-4666-95d6-d9067f3dd3fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:03:24.999800 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db75a4a-b009-4666-95d6-d9067f3dd3fa None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:25.017612 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02dadcf-e1c3-46e3-82f9-2b1813ba3e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:25.018714 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02dadcf-e1c3-46e3-82f9-2b1813ba3e53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:25.019162 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02dadcf-e1c3-46e3-82f9-2b1813ba3e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:25.019492 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02dadcf-e1c3-46e3-82f9-2b1813ba3e53 admin admin] PATH_INFO: `/v3/projects/de5fab60406c42d3be1c783e6e774bf1` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:25.020917 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02dadcf-e1c3-46e3-82f9-2b1813ba3e53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de5fab60406c42d3be1c783e6e774bf1)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:25.022914 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02dadcf-e1c3-46e3-82f9-2b1813ba3e53 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:25.107100 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-b02dadcf-e1c3-46e3-82f9-2b1813ba3e53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de5fab60406c42d3be1c783e6e774bf1', 'request_id': 'req-b02dadcf-e1c3-46e3-82f9-2b1813ba3e53'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:25.114178 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-b02dadcf-e1c3-46e3-82f9-2b1813ba3e53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de5fab60406c42d3be1c783e6e774bf1', 'request_id': 'req-b02dadcf-e1c3-46e3-82f9-2b1813ba3e53'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:25.124809 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-b02dadcf-e1c3-46e3-82f9-2b1813ba3e53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de5fab60406c42d3be1c783e6e774bf1', 'request_id': 'req-b02dadcf-e1c3-46e3-82f9-2b1813ba3e53'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:25.148617 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-b02dadcf-e1c3-46e3-82f9-2b1813ba3e53 admin admin] The token cache is being invalidate because project de5fab60406c42d3be1c783e6e774bf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 02:03:25.149058 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-b02dadcf-e1c3-46e3-82f9-2b1813ba3e53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:25.149839 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-b02dadcf-e1c3-46e3-82f9-2b1813ba3e53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:25.152336 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1304/2578] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 02:03:24 2022] DELETE /identity/v3/projects/de5fab60406c42d3be1c783e6e774bf1 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:03:25.158633 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d3f6e-7224-4431-90da-3cc58efe083c None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:25.159554 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d3f6e-7224-4431-90da-3cc58efe083c None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:25.159976 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d3f6e-7224-4431-90da-3cc58efe083c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:25.169914 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:03:25.183232 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-367d3f6e-7224-4431-90da-3cc58efe083c None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:03:25.184465 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-367d3f6e-7224-4431-90da-3cc58efe083c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:03:25.249936 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1275/2579] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:03:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:03:25.257814 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd73b9-13bf-451e-991d-869e41d5db99 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:25.258098 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd73b9-13bf-451e-991d-869e41d5db99 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:25.258556 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd73b9-13bf-451e-991d-869e41d5db99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:25.284303 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-d0cd73b9-13bf-451e-991d-869e41d5db99 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:03:25.285705 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-d0cd73b9-13bf-451e-991d-869e41d5db99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:03:25.295919 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1305/2580] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:03:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:03:25.299998 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367d3f6e-7224-4431-90da-3cc58efe083c None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:25.311649 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367d3f6e-7224-4431-90da-3cc58efe083c None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:25.317007 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d9c0c9-2d9a-4a5a-9ba2-2c5d95ccb0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:25.317749 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d9c0c9-2d9a-4a5a-9ba2-2c5d95ccb0e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:25.318031 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d9c0c9-2d9a-4a5a-9ba2-2c5d95ccb0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:25.318304 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d9c0c9-2d9a-4a5a-9ba2-2c5d95ccb0e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:25.319257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d9c0c9-2d9a-4a5a-9ba2-2c5d95ccb0e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 02:03:25.319554 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d9c0c9-2d9a-4a5a-9ba2-2c5d95ccb0e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:25.321173 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d9c0c9-2d9a-4a5a-9ba2-2c5d95ccb0e8 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:25.336070 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1276/2581] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 02:03:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 02:03:25.340565 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd73b9-13bf-451e-991d-869e41d5db99 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:25.350650 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd73b9-13bf-451e-991d-869e41d5db99 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:25.356133 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2999a1c-b5bf-49aa-af1b-7315bfb0b9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:25.356966 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2999a1c-b5bf-49aa-af1b-7315bfb0b9ff admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:25.357355 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2999a1c-b5bf-49aa-af1b-7315bfb0b9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:25.357729 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2999a1c-b5bf-49aa-af1b-7315bfb0b9ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:25.358529 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2999a1c-b5bf-49aa-af1b-7315bfb0b9ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:25.360725 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2999a1c-b5bf-49aa-af1b-7315bfb0b9ff admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:25.395596 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1306/2582] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 02:03:25 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:03:25.396852 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d9c0c9-2d9a-4a5a-9ba2-2c5d95ccb0e8 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:25.408210 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d9c0c9-2d9a-4a5a-9ba2-2c5d95ccb0e8 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:25.471371 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7b812c-eeee-40de-af57-ed2560af4a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:25.471896 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b812c-eeee-40de-af57-ed2560af4a74 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:25.472293 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b812c-eeee-40de-af57-ed2560af4a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:25.472603 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b812c-eeee-40de-af57-ed2560af4a74 admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:25.473324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7b812c-eeee-40de-af57-ed2560af4a74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:25.475275 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7b812c-eeee-40de-af57-ed2560af4a74 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:25.496154 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1277/2583] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 02:03:25 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:03:25.500151 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2999a1c-b5bf-49aa-af1b-7315bfb0b9ff admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:25.512461 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2999a1c-b5bf-49aa-af1b-7315bfb0b9ff admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:25.518178 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae738a1-7d10-43c7-9b56-657df4234cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:25.518849 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae738a1-7d10-43c7-9b56-657df4234cf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:25.519128 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae738a1-7d10-43c7-9b56-657df4234cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:25.519432 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae738a1-7d10-43c7-9b56-657df4234cf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:25.520164 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae738a1-7d10-43c7-9b56-657df4234cf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:25.521782 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae738a1-7d10-43c7-9b56-657df4234cf8 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:25.533492 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1307/2584] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:03:25.537632 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7b812c-eeee-40de-af57-ed2560af4a74 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:25.548411 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7b812c-eeee-40de-af57-ed2560af4a74 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:25.554745 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ef4940-c5fb-461b-88c2-ce78335cd903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:25.555527 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ef4940-c5fb-461b-88c2-ce78335cd903 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:25.555921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ef4940-c5fb-461b-88c2-ce78335cd903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:25.556287 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ef4940-c5fb-461b-88c2-ce78335cd903 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:25.557095 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ef4940-c5fb-461b-88c2-ce78335cd903 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:25.558763 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ef4940-c5fb-461b-88c2-ce78335cd903 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:25.572608 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1278/2585] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 02:03:25.573963 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae738a1-7d10-43c7-9b56-657df4234cf8 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:25.584691 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae738a1-7d10-43c7-9b56-657df4234cf8 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:25.590323 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a053081b-05e1-4467-85ae-b33ecdf8518a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:25.591274 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a053081b-05e1-4467-85ae-b33ecdf8518a admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:25.591530 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a053081b-05e1-4467-85ae-b33ecdf8518a admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:25.591803 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a053081b-05e1-4467-85ae-b33ecdf8518a admin admin] PATH_INFO: `/v3/projects/734fd8c4854d4fe99d158cf249181488/users/25d30d06556f4b7badff1e49b2ba49c1/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:25.607262 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a053081b-05e1-4467-85ae-b33ecdf8518a admin admin] RBAC: Authorizing `identity:create_grant(project_id=734fd8c4854d4fe99d158cf249181488, user_id=25d30d06556f4b7badff1e49b2ba49c1, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:25.610516 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a053081b-05e1-4467-85ae-b33ecdf8518a admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:25.624708 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1308/2586] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 02:03:25 2022] PUT /identity/v3/projects/734fd8c4854d4fe99d158cf249181488/users/25d30d06556f4b7badff1e49b2ba49c1/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 02:03:25.634896 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa133f20-62ec-4761-8c3f-635ebde833a9 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:25.635171 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa133f20-62ec-4761-8c3f-635ebde833a9 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:25.635682 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa133f20-62ec-4761-8c3f-635ebde833a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:25.664990 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-fa133f20-62ec-4761-8c3f-635ebde833a9 None None] MFA Rules not processed for user `25d30d06556f4b7badff1e49b2ba49c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:03:25.666528 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-fa133f20-62ec-4761-8c3f-635ebde833a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:03:25.741835 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:03:25.754366 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1279/2587] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:03:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:03:25.774567 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a053081b-05e1-4467-85ae-b33ecdf8518a admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:25.776796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-a053081b-05e1-4467-85ae-b33ecdf8518a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:03:25.850030 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a053081b-05e1-4467-85ae-b33ecdf8518a admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:25.854344 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3cad5a-84dd-4f20-92a7-2c099f0cd310 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:25.855149 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3cad5a-84dd-4f20-92a7-2c099f0cd310 service cinder] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:25.855576 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3cad5a-84dd-4f20-92a7-2c099f0cd310 service cinder] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:25.856403 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3cad5a-84dd-4f20-92a7-2c099f0cd310 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:25.871840 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3cad5a-84dd-4f20-92a7-2c099f0cd310 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:25.874377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3cad5a-84dd-4f20-92a7-2c099f0cd310 service cinder] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:25.887565 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1309/2588] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:03:25 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:03:27.351961 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c04aec0-c090-428b-bf93-540758f0b184 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:27.352559 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c04aec0-c090-428b-bf93-540758f0b184 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:27.352661 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c04aec0-c090-428b-bf93-540758f0b184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:27.370870 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-4c04aec0-c090-428b-bf93-540758f0b184 None None] MFA Rules not processed for user `25d30d06556f4b7badff1e49b2ba49c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:03:27.372194 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-4c04aec0-c090-428b-bf93-540758f0b184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:03:27.379516 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1280/2589] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:03:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:03:35.883158 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=81,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:03:37.365674 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=3 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:03:43.329693 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52436f5c-7ea7-4850-aa8a-09b687be86d0 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:43.330193 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52436f5c-7ea7-4850-aa8a-09b687be86d0 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:43.330292 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52436f5c-7ea7-4850-aa8a-09b687be86d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:43.348609 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-52436f5c-7ea7-4850-aa8a-09b687be86d0 None None] MFA Rules not processed for user `168e6181ad22460990f1f7361b483364`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:03:43.349847 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-52436f5c-7ea7-4850-aa8a-09b687be86d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:03:43.407887 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1310/2590] 10.222.0.26 () {58 vars in 1009 bytes} [Thu Jun 23 02:03:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:03:43.433429 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c04aec0-c090-428b-bf93-540758f0b184 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:43.435884 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-4c04aec0-c090-428b-bf93-540758f0b184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:03:43.520181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c04aec0-c090-428b-bf93-540758f0b184 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:43.528041 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e57b3b0-8c50-4e0a-96e0-b2e41e62691b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28e59688485241c4b6d51f99bbb3d059', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:43.528943 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57b3b0-8c50-4e0a-96e0-b2e41e62691b service swift] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:43.529280 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57b3b0-8c50-4e0a-96e0-b2e41e62691b service swift] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:43.529603 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57b3b0-8c50-4e0a-96e0-b2e41e62691b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:43.539119 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e57b3b0-8c50-4e0a-96e0-b2e41e62691b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:43.540749 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e57b3b0-8c50-4e0a-96e0-b2e41e62691b service swift] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:43.549380 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1281/2591] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Jun 23 02:03:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 02:03:44.085959 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440a587-87ad-4707-b830-c1e76941ffee None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:44.086244 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440a587-87ad-4707-b830-c1e76941ffee None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:44.086515 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440a587-87ad-4707-b830-c1e76941ffee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:44.106662 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-f440a587-87ad-4707-b830-c1e76941ffee None None] MFA Rules not processed for user `168e6181ad22460990f1f7361b483364`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:03:44.107907 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-f440a587-87ad-4707-b830-c1e76941ffee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:03:44.115538 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1311/2592] 10.222.0.26 () {58 vars in 967 bytes} [Thu Jun 23 02:03:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:03:44.128319 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e57b3b0-8c50-4e0a-96e0-b2e41e62691b service swift] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:44.140082 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e57b3b0-8c50-4e0a-96e0-b2e41e62691b service swift] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:44.148847 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e52e531-886f-414a-ac0f-c33aa3a3c685 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28e59688485241c4b6d51f99bbb3d059', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:44.149560 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e52e531-886f-414a-ac0f-c33aa3a3c685 service swift] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:44.149924 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e52e531-886f-414a-ac0f-c33aa3a3c685 service swift] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:44.150258 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e52e531-886f-414a-ac0f-c33aa3a3c685 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:44.159183 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e52e531-886f-414a-ac0f-c33aa3a3c685 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:44.160820 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e52e531-886f-414a-ac0f-c33aa3a3c685 service swift] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:44.170355 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1282/2593] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Jun 23 02:03:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 23 02:03:44.377868 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58606353-4f93-4d76-b124-4c960361a273 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:44.378352 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58606353-4f93-4d76-b124-4c960361a273 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:44.378444 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58606353-4f93-4d76-b124-4c960361a273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:44.396756 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-58606353-4f93-4d76-b124-4c960361a273 None None] MFA Rules not processed for user `168e6181ad22460990f1f7361b483364`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:03:44.397969 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-58606353-4f93-4d76-b124-4c960361a273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:03:44.405752 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1312/2594] 10.222.0.26 () {58 vars in 1009 bytes} [Thu Jun 23 02:03:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 02:03:44.418319 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e52e531-886f-414a-ac0f-c33aa3a3c685 service swift] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:44.429191 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e52e531-886f-414a-ac0f-c33aa3a3c685 service swift] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:44.435562 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6865b27-ccd0-4adb-854e-a67dbafad589 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28e59688485241c4b6d51f99bbb3d059', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:44.436203 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6865b27-ccd0-4adb-854e-a67dbafad589 service swift] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:44.436505 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6865b27-ccd0-4adb-854e-a67dbafad589 service swift] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:44.436959 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6865b27-ccd0-4adb-854e-a67dbafad589 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:44.445843 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6865b27-ccd0-4adb-854e-a67dbafad589 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:44.447461 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6865b27-ccd0-4adb-854e-a67dbafad589 service swift] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:44.455736 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1283/2595] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Jun 23 02:03:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 02:03:44.788979 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18858c50-3ea2-45e5-bc60-01f107594b01 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:44.789261 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18858c50-3ea2-45e5-bc60-01f107594b01 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:44.789699 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18858c50-3ea2-45e5-bc60-01f107594b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:44.814384 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-18858c50-3ea2-45e5-bc60-01f107594b01 None None] MFA Rules not processed for user `168e6181ad22460990f1f7361b483364`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:03:44.817127 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-18858c50-3ea2-45e5-bc60-01f107594b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:03:44.826361 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1313/2596] 10.222.0.26 () {58 vars in 967 bytes} [Thu Jun 23 02:03:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:03:44.839625 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6865b27-ccd0-4adb-854e-a67dbafad589 service swift] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:44.850823 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6865b27-ccd0-4adb-854e-a67dbafad589 service swift] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:44.856104 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28a0eb5-4a7e-4a55-8d48-2ff2ab4eb8bb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28e59688485241c4b6d51f99bbb3d059', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:44.856877 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a0eb5-4a7e-4a55-8d48-2ff2ab4eb8bb service swift] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:44.857165 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a0eb5-4a7e-4a55-8d48-2ff2ab4eb8bb service swift] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:44.857441 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a0eb5-4a7e-4a55-8d48-2ff2ab4eb8bb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:44.865869 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28a0eb5-4a7e-4a55-8d48-2ff2ab4eb8bb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:44.867516 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28a0eb5-4a7e-4a55-8d48-2ff2ab4eb8bb service swift] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:44.875884 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1284/2597] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Jun 23 02:03:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 02:03:47.055232 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18858c50-3ea2-45e5-bc60-01f107594b01 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:47.125289 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18858c50-3ea2-45e5-bc60-01f107594b01 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:47.131655 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba782d8-16af-467d-886b-2a895980c8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:47.132603 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba782d8-16af-467d-886b-2a895980c8ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:47.132877 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba782d8-16af-467d-886b-2a895980c8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:47.133147 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba782d8-16af-467d-886b-2a895980c8ef admin admin] PATH_INFO: `/v3/users/25d30d06556f4b7badff1e49b2ba49c1` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:47.134553 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba782d8-16af-467d-886b-2a895980c8ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=25d30d06556f4b7badff1e49b2ba49c1)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:47.136604 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba782d8-16af-467d-886b-2a895980c8ef admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:47.200576 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-0ba782d8-16af-467d-886b-2a895980c8ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25d30d06556f4b7badff1e49b2ba49c1'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:47.218131 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-0ba782d8-16af-467d-886b-2a895980c8ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25d30d06556f4b7badff1e49b2ba49c1'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:47.228042 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-0ba782d8-16af-467d-886b-2a895980c8ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25d30d06556f4b7badff1e49b2ba49c1'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:47.236447 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-0ba782d8-16af-467d-886b-2a895980c8ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25d30d06556f4b7badff1e49b2ba49c1'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:47.237214 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-0ba782d8-16af-467d-886b-2a895980c8ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25d30d06556f4b7badff1e49b2ba49c1'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:47.244314 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1314/2598] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 02:03:47 2022] DELETE /identity/v3/users/25d30d06556f4b7badff1e49b2ba49c1 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:03:47.249177 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e28a6-42cd-48e3-8b45-5aa8923db211 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:47.249711 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e28a6-42cd-48e3-8b45-5aa8923db211 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:47.249879 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e28a6-42cd-48e3-8b45-5aa8923db211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:47.268545 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-f05e28a6-42cd-48e3-8b45-5aa8923db211 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:03:47.269809 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-f05e28a6-42cd-48e3-8b45-5aa8923db211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:03:47.339064 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1285/2599] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:03:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:03:47.349096 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba782d8-16af-467d-886b-2a895980c8ef admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:47.350643 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-0ba782d8-16af-467d-886b-2a895980c8ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:03:47.357531 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:03:47.410640 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba782d8-16af-467d-886b-2a895980c8ef admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:47.415341 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c1956-af34-4380-af85-a36d37cda73e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:47.416296 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c1956-af34-4380-af85-a36d37cda73e service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:47.416569 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c1956-af34-4380-af85-a36d37cda73e service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:47.416839 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c1956-af34-4380-af85-a36d37cda73e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:47.426377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316c1956-af34-4380-af85-a36d37cda73e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:47.427868 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316c1956-af34-4380-af85-a36d37cda73e service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:47.437501 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1315/2600] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:03:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:03:47.634703 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e28a6-42cd-48e3-8b45-5aa8923db211 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:47.636462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-f05e28a6-42cd-48e3-8b45-5aa8923db211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:03:47.650304 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e28a6-42cd-48e3-8b45-5aa8923db211 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:47.655841 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74d926a-05e6-4fce-b51c-cac27b6296a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:47.656725 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d926a-05e6-4fce-b51c-cac27b6296a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:47.657003 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d926a-05e6-4fce-b51c-cac27b6296a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:47.657275 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d926a-05e6-4fce-b51c-cac27b6296a3 admin admin] PATH_INFO: `/v3/projects/734fd8c4854d4fe99d158cf249181488` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:47.658506 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74d926a-05e6-4fce-b51c-cac27b6296a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=734fd8c4854d4fe99d158cf249181488)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:47.660476 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74d926a-05e6-4fce-b51c-cac27b6296a3 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:47.705918 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:03:47.709221 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-b74d926a-05e6-4fce-b51c-cac27b6296a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '734fd8c4854d4fe99d158cf249181488', 'request_id': 'req-b74d926a-05e6-4fce-b51c-cac27b6296a3'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:47.715720 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-b74d926a-05e6-4fce-b51c-cac27b6296a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '734fd8c4854d4fe99d158cf249181488', 'request_id': 'req-b74d926a-05e6-4fce-b51c-cac27b6296a3'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:47.724320 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-b74d926a-05e6-4fce-b51c-cac27b6296a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '734fd8c4854d4fe99d158cf249181488', 'request_id': 'req-b74d926a-05e6-4fce-b51c-cac27b6296a3'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:47.736473 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-b74d926a-05e6-4fce-b51c-cac27b6296a3 admin admin] The token cache is being invalidate because project 734fd8c4854d4fe99d158cf249181488 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 02:03:47.736766 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-b74d926a-05e6-4fce-b51c-cac27b6296a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:47.737542 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-b74d926a-05e6-4fce-b51c-cac27b6296a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:47.739173 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1286/2601] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 02:03:47 2022] DELETE /identity/v3/projects/734fd8c4854d4fe99d158cf249181488 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:03:49.491103 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c1956-af34-4380-af85-a36d37cda73e service neutron] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:49.492721 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-316c1956-af34-4380-af85-a36d37cda73e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:03:49.505100 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c1956-af34-4380-af85-a36d37cda73e service neutron] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:49.556551 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb677e6-1e14-45a1-9f67-a2ef94bfc69b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:49.557112 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb677e6-1e14-45a1-9f67-a2ef94bfc69b service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:49.557393 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb677e6-1e14-45a1-9f67-a2ef94bfc69b service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:49.557661 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb677e6-1e14-45a1-9f67-a2ef94bfc69b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:49.559774 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-3fb677e6-1e14-45a1-9f67-a2ef94bfc69b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:03:49.629795 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb677e6-1e14-45a1-9f67-a2ef94bfc69b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:49.631242 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb677e6-1e14-45a1-9f67-a2ef94bfc69b service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:49.642391 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1316/2602] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:03:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:03:52.748487 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74d926a-05e6-4fce-b51c-cac27b6296a3 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:52.750584 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-b74d926a-05e6-4fce-b51c-cac27b6296a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:03:52.764308 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74d926a-05e6-4fce-b51c-cac27b6296a3 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:52.769116 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e0e68c-751e-4f2d-b72a-ff21e02c4b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:52.770027 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e0e68c-751e-4f2d-b72a-ff21e02c4b94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:52.770303 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e0e68c-751e-4f2d-b72a-ff21e02c4b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:52.770596 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e0e68c-751e-4f2d-b72a-ff21e02c4b94 admin admin] PATH_INFO: `/v3/users/0287bd1260d24e5a94c00367b23c7a84` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:52.771880 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e0e68c-751e-4f2d-b72a-ff21e02c4b94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0287bd1260d24e5a94c00367b23c7a84)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:52.774576 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e0e68c-751e-4f2d-b72a-ff21e02c4b94 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:52.823092 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-37e0e68c-751e-4f2d-b72a-ff21e02c4b94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0287bd1260d24e5a94c00367b23c7a84'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:52.834860 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-37e0e68c-751e-4f2d-b72a-ff21e02c4b94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0287bd1260d24e5a94c00367b23c7a84'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:52.843387 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-37e0e68c-751e-4f2d-b72a-ff21e02c4b94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0287bd1260d24e5a94c00367b23c7a84'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:52.850440 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-37e0e68c-751e-4f2d-b72a-ff21e02c4b94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0287bd1260d24e5a94c00367b23c7a84'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:52.851092 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-37e0e68c-751e-4f2d-b72a-ff21e02c4b94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0287bd1260d24e5a94c00367b23c7a84'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:52.857176 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1287/2603] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 02:03:52 2022] DELETE /identity/v3/users/0287bd1260d24e5a94c00367b23c7a84 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:03:53.035287 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb677e6-1e14-45a1-9f67-a2ef94bfc69b service neutron] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:53.046224 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb677e6-1e14-45a1-9f67-a2ef94bfc69b service neutron] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:53.098945 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:03:53.105526 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f6c5d7-dba6-407f-a9d7-bb2cc706dddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:53.106428 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6c5d7-dba6-407f-a9d7-bb2cc706dddd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:53.106707 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6c5d7-dba6-407f-a9d7-bb2cc706dddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:53.106980 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6c5d7-dba6-407f-a9d7-bb2cc706dddd admin admin] PATH_INFO: `/v3/projects/d0648b2071364ae09ba6d7a1d45c762c` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:53.108311 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f6c5d7-dba6-407f-a9d7-bb2cc706dddd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0648b2071364ae09ba6d7a1d45c762c)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:53.110271 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f6c5d7-dba6-407f-a9d7-bb2cc706dddd admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:53.159004 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-70f6c5d7-dba6-407f-a9d7-bb2cc706dddd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0648b2071364ae09ba6d7a1d45c762c', 'request_id': 'req-70f6c5d7-dba6-407f-a9d7-bb2cc706dddd'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:53.165344 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-70f6c5d7-dba6-407f-a9d7-bb2cc706dddd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0648b2071364ae09ba6d7a1d45c762c', 'request_id': 'req-70f6c5d7-dba6-407f-a9d7-bb2cc706dddd'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:53.174581 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-70f6c5d7-dba6-407f-a9d7-bb2cc706dddd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0648b2071364ae09ba6d7a1d45c762c', 'request_id': 'req-70f6c5d7-dba6-407f-a9d7-bb2cc706dddd'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:53.185207 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-70f6c5d7-dba6-407f-a9d7-bb2cc706dddd admin admin] The token cache is being invalidate because project d0648b2071364ae09ba6d7a1d45c762c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 02:03:53.185490 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-70f6c5d7-dba6-407f-a9d7-bb2cc706dddd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:53.186113 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-70f6c5d7-dba6-407f-a9d7-bb2cc706dddd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:53.187828 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1317/2604] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 02:03:53 2022] DELETE /identity/v3/projects/d0648b2071364ae09ba6d7a1d45c762c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:03:53.192956 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd612dd-d819-4861-b3ae-19ac4cfcf8e0 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:53.193241 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd612dd-d819-4861-b3ae-19ac4cfcf8e0 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:53.193514 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd612dd-d819-4861-b3ae-19ac4cfcf8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:53.212062 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-7fd612dd-d819-4861-b3ae-19ac4cfcf8e0 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:03:53.213269 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-7fd612dd-d819-4861-b3ae-19ac4cfcf8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:03:53.275722 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1288/2605] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:03:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:03:53.284421 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb25ad8-aa09-4511-8870-374d57966964 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:53.284707 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb25ad8-aa09-4511-8870-374d57966964 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:53.285008 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb25ad8-aa09-4511-8870-374d57966964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:53.305085 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-2bb25ad8-aa09-4511-8870-374d57966964 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:03:53.306281 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-2bb25ad8-aa09-4511-8870-374d57966964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:03:53.314788 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1318/2606] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:03:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:03:53.319256 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd612dd-d819-4861-b3ae-19ac4cfcf8e0 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:53.329431 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd612dd-d819-4861-b3ae-19ac4cfcf8e0 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:53.335233 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66407310-b4c7-417f-8e6f-f214ab5a3351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:53.335972 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66407310-b4c7-417f-8e6f-f214ab5a3351 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:53.336253 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66407310-b4c7-417f-8e6f-f214ab5a3351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:53.336523 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66407310-b4c7-417f-8e6f-f214ab5a3351 admin admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:53.337255 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66407310-b4c7-417f-8e6f-f214ab5a3351 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 02:03:53.337563 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66407310-b4c7-417f-8e6f-f214ab5a3351 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:53.339164 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66407310-b4c7-417f-8e6f-f214ab5a3351 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:53.351750 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1289/2607] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 02:03:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 02:03:53.356131 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb25ad8-aa09-4511-8870-374d57966964 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:53.365859 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb25ad8-aa09-4511-8870-374d57966964 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:53.371848 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4633c6ba-f30d-4548-83a4-534ea8e50557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:53.372490 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633c6ba-f30d-4548-83a4-534ea8e50557 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:53.372768 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633c6ba-f30d-4548-83a4-534ea8e50557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:53.373034 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633c6ba-f30d-4548-83a4-534ea8e50557 admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:53.373716 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4633c6ba-f30d-4548-83a4-534ea8e50557 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:53.375744 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4633c6ba-f30d-4548-83a4-534ea8e50557 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:53.403787 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1319/2608] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 02:03:53 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:03:53.408279 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66407310-b4c7-417f-8e6f-f214ab5a3351 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:53.419051 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66407310-b4c7-417f-8e6f-f214ab5a3351 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:53.450231 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:03:53.478261 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2c4104-2c61-4880-b7f2-3bdfa3d26c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:53.478921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c4104-2c61-4880-b7f2-3bdfa3d26c37 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:53.479204 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c4104-2c61-4880-b7f2-3bdfa3d26c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:53.479515 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c4104-2c61-4880-b7f2-3bdfa3d26c37 admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:53.480192 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2c4104-2c61-4880-b7f2-3bdfa3d26c37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:53.482147 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2c4104-2c61-4880-b7f2-3bdfa3d26c37 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:53.503643 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1290/2609] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 02:03:53 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:03:53.509179 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4633c6ba-f30d-4548-83a4-534ea8e50557 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:53.519399 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4633c6ba-f30d-4548-83a4-534ea8e50557 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:53.524480 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03afc68-2c29-410e-bbfb-733fc0faca6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:53.525122 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03afc68-2c29-410e-bbfb-733fc0faca6d admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:53.525397 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03afc68-2c29-410e-bbfb-733fc0faca6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:53.525663 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03afc68-2c29-410e-bbfb-733fc0faca6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:53.526442 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03afc68-2c29-410e-bbfb-733fc0faca6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:53.528033 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03afc68-2c29-410e-bbfb-733fc0faca6d admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:53.538612 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1320/2610] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:03:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:03:53.542990 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2c4104-2c61-4880-b7f2-3bdfa3d26c37 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:53.553483 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2c4104-2c61-4880-b7f2-3bdfa3d26c37 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:53.559018 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b314feef-2c83-4b4e-9de1-9ec473903701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:53.559708 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314feef-2c83-4b4e-9de1-9ec473903701 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:53.559990 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314feef-2c83-4b4e-9de1-9ec473903701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:53.560262 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314feef-2c83-4b4e-9de1-9ec473903701 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:53.560995 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b314feef-2c83-4b4e-9de1-9ec473903701 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:53.562620 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b314feef-2c83-4b4e-9de1-9ec473903701 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:53.572873 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1291/2611] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:03:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:03:53.577021 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03afc68-2c29-410e-bbfb-733fc0faca6d admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:53.587461 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03afc68-2c29-410e-bbfb-733fc0faca6d admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:53.592798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93736e81-fdea-418f-8bc8-f03ff3b2337d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:53.593677 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93736e81-fdea-418f-8bc8-f03ff3b2337d admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:53.593954 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93736e81-fdea-418f-8bc8-f03ff3b2337d admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:53.594224 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93736e81-fdea-418f-8bc8-f03ff3b2337d admin admin] PATH_INFO: `/v3/projects/6e828073368644c18d654a07f3e56a52/users/a9cbd7f2a3ba4894b2059d79a043416a/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:53.610382 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93736e81-fdea-418f-8bc8-f03ff3b2337d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e828073368644c18d654a07f3e56a52, user_id=a9cbd7f2a3ba4894b2059d79a043416a, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:53.613086 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93736e81-fdea-418f-8bc8-f03ff3b2337d admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:53.628254 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1321/2612] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 02:03:53 2022] PUT /identity/v3/projects/6e828073368644c18d654a07f3e56a52/users/a9cbd7f2a3ba4894b2059d79a043416a/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 02:03:53.639379 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc4b0c8-4c3c-49c9-a39f-27d12a2cd87c None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:53.639726 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc4b0c8-4c3c-49c9-a39f-27d12a2cd87c None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:53.640005 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc4b0c8-4c3c-49c9-a39f-27d12a2cd87c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:53.668340 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-1bc4b0c8-4c3c-49c9-a39f-27d12a2cd87c None None] MFA Rules not processed for user `a9cbd7f2a3ba4894b2059d79a043416a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:03:53.669617 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-1bc4b0c8-4c3c-49c9-a39f-27d12a2cd87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:03:53.735418 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1292/2613] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:03:53 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:03:53.740070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93736e81-fdea-418f-8bc8-f03ff3b2337d admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:53.806483 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93736e81-fdea-418f-8bc8-f03ff3b2337d admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:53.811586 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d055e9-8b52-4e4b-bacf-f287b08af5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:53.812235 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d055e9-8b52-4e4b-bacf-f287b08af5ae admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:53.812521 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d055e9-8b52-4e4b-bacf-f287b08af5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:53.812792 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d055e9-8b52-4e4b-bacf-f287b08af5ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:53.813474 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d055e9-8b52-4e4b-bacf-f287b08af5ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:53.815404 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d055e9-8b52-4e4b-bacf-f287b08af5ae admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:53.842618 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1322/2614] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 02:03:53 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:03:53.848552 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc4b0c8-4c3c-49c9-a39f-27d12a2cd87c None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:53.862345 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc4b0c8-4c3c-49c9-a39f-27d12a2cd87c None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:53.922238 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a4eef3-f317-41ba-9925-004633055794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:53.922909 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a4eef3-f317-41ba-9925-004633055794 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:53.923188 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a4eef3-f317-41ba-9925-004633055794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:53.923473 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a4eef3-f317-41ba-9925-004633055794 admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:53.924158 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a4eef3-f317-41ba-9925-004633055794 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:53.926069 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a4eef3-f317-41ba-9925-004633055794 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:53.947396 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1293/2615] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 02:03:53 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:03:53.951464 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d055e9-8b52-4e4b-bacf-f287b08af5ae admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:53.961954 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d055e9-8b52-4e4b-bacf-f287b08af5ae admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:53.966944 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c101f-1242-4976-95ff-c965634ca97d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:53.967724 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c101f-1242-4976-95ff-c965634ca97d admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:53.968011 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c101f-1242-4976-95ff-c965634ca97d admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:53.968286 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c101f-1242-4976-95ff-c965634ca97d admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:53.969022 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9c101f-1242-4976-95ff-c965634ca97d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:53.970630 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9c101f-1242-4976-95ff-c965634ca97d admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:53.972792 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:03:53.982668 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1323/2616] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:03:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:03:53.986724 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a4eef3-f317-41ba-9925-004633055794 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:53.997120 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a4eef3-f317-41ba-9925-004633055794 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:54.002185 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdea71a-aa85-4594-b1cf-24faf193413a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:54.002909 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdea71a-aa85-4594-b1cf-24faf193413a admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:54.003189 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdea71a-aa85-4594-b1cf-24faf193413a admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:54.003497 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdea71a-aa85-4594-b1cf-24faf193413a admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:54.004219 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdea71a-aa85-4594-b1cf-24faf193413a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:54.005822 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdea71a-aa85-4594-b1cf-24faf193413a admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:54.015841 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1294/2617] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:03:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:03:54.019938 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c101f-1242-4976-95ff-c965634ca97d admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:54.029932 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c101f-1242-4976-95ff-c965634ca97d admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:54.034803 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf44ae4-b772-4f14-8fef-e799a3f745fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:54.035712 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf44ae4-b772-4f14-8fef-e799a3f745fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:54.036272 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf44ae4-b772-4f14-8fef-e799a3f745fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:54.036272 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf44ae4-b772-4f14-8fef-e799a3f745fe admin admin] PATH_INFO: `/v3/projects/6b44be1decb34d81bbe710d73ba6bc50/users/ce1ab5284203403c8bcffbcf3a26c88f/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:54.051587 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf44ae4-b772-4f14-8fef-e799a3f745fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b44be1decb34d81bbe710d73ba6bc50, user_id=ce1ab5284203403c8bcffbcf3a26c88f, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:54.054227 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf44ae4-b772-4f14-8fef-e799a3f745fe admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:54.069054 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1324/2618] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 02:03:54 2022] PUT /identity/v3/projects/6b44be1decb34d81bbe710d73ba6bc50/users/ce1ab5284203403c8bcffbcf3a26c88f/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:03:54.079656 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6825e5a-8bc1-4ac1-b83e-9b2cdd475df9 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:54.079974 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6825e5a-8bc1-4ac1-b83e-9b2cdd475df9 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:54.080234 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6825e5a-8bc1-4ac1-b83e-9b2cdd475df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:54.109707 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-d6825e5a-8bc1-4ac1-b83e-9b2cdd475df9 None None] MFA Rules not processed for user `ce1ab5284203403c8bcffbcf3a26c88f`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:03:54.111745 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-d6825e5a-8bc1-4ac1-b83e-9b2cdd475df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:03:54.122666 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:03:54.182289 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1295/2619] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:03:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:03:54.187084 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf44ae4-b772-4f14-8fef-e799a3f745fe admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:54.258772 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf44ae4-b772-4f14-8fef-e799a3f745fe admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:54.264879 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58206067-6f55-4de7-815d-59f74bf1645d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:54.265551 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58206067-6f55-4de7-815d-59f74bf1645d admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:54.265833 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58206067-6f55-4de7-815d-59f74bf1645d admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:54.266088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58206067-6f55-4de7-815d-59f74bf1645d admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:54.266786 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58206067-6f55-4de7-815d-59f74bf1645d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:54.268786 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58206067-6f55-4de7-815d-59f74bf1645d admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:54.305144 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1325/2620] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 02:03:54 2022] POST /identity/v3/projects => generated 377 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:03:54.309438 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6825e5a-8bc1-4ac1-b83e-9b2cdd475df9 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:54.320294 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6825e5a-8bc1-4ac1-b83e-9b2cdd475df9 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:54.379065 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8719c502-2e35-4d28-85db-ad162c8aaf5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:54.379767 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8719c502-2e35-4d28-85db-ad162c8aaf5c admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:54.380049 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8719c502-2e35-4d28-85db-ad162c8aaf5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:54.380326 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8719c502-2e35-4d28-85db-ad162c8aaf5c admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:54.381023 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8719c502-2e35-4d28-85db-ad162c8aaf5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:54.382943 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8719c502-2e35-4d28-85db-ad162c8aaf5c admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:54.409606 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1296/2621] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 02:03:54 2022] POST /identity/v3/users => generated 295 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 02:03:54.416327 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58206067-6f55-4de7-815d-59f74bf1645d admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:54.428822 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58206067-6f55-4de7-815d-59f74bf1645d admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:54.434367 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56190c30-c675-4211-b644-7c2b868e0473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:54.435023 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56190c30-c675-4211-b644-7c2b868e0473 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:54.435341 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56190c30-c675-4211-b644-7c2b868e0473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:54.435612 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56190c30-c675-4211-b644-7c2b868e0473 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:54.436345 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56190c30-c675-4211-b644-7c2b868e0473 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:54.437983 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56190c30-c675-4211-b644-7c2b868e0473 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:54.449291 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1326/2622] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:03:54.453842 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8719c502-2e35-4d28-85db-ad162c8aaf5c admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:54.464500 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8719c502-2e35-4d28-85db-ad162c8aaf5c admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:54.469553 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb86123-768d-4c65-aca8-3a7ae9dc7e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:54.470425 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb86123-768d-4c65-aca8-3a7ae9dc7e78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:54.470717 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb86123-768d-4c65-aca8-3a7ae9dc7e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:54.470989 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb86123-768d-4c65-aca8-3a7ae9dc7e78 admin admin] PATH_INFO: `/v3/projects/d9d5b422d73147ffbcc7e623a15c38a5/users/cfe8512bb42f402c9c61480f809ca759/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:54.486436 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb86123-768d-4c65-aca8-3a7ae9dc7e78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9d5b422d73147ffbcc7e623a15c38a5, user_id=cfe8512bb42f402c9c61480f809ca759, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:54.489216 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb86123-768d-4c65-aca8-3a7ae9dc7e78 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:54.504570 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1297/2623] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 02:03:54 2022] PUT /identity/v3/projects/d9d5b422d73147ffbcc7e623a15c38a5/users/cfe8512bb42f402c9c61480f809ca759/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:03:54.508758 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56190c30-c675-4211-b644-7c2b868e0473 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:54.518612 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56190c30-c675-4211-b644-7c2b868e0473 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:54.580467 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e3a3a-1cb3-45e1-8270-4f77b1506751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:54.581295 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e3a3a-1cb3-45e1-8270-4f77b1506751 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:54.581751 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e3a3a-1cb3-45e1-8270-4f77b1506751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:54.582177 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e3a3a-1cb3-45e1-8270-4f77b1506751 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:54.583406 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3e3a3a-1cb3-45e1-8270-4f77b1506751 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:54.586330 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3e3a3a-1cb3-45e1-8270-4f77b1506751 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:54.604083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1327/2624] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 02:03:54.610085 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb86123-768d-4c65-aca8-3a7ae9dc7e78 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:54.621514 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb86123-768d-4c65-aca8-3a7ae9dc7e78 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:54.627153 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62508d92-fbb9-4f22-9c1d-f08db7812d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:54.628150 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62508d92-fbb9-4f22-9c1d-f08db7812d6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:54.628499 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62508d92-fbb9-4f22-9c1d-f08db7812d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:54.628839 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62508d92-fbb9-4f22-9c1d-f08db7812d6b admin admin] PATH_INFO: `/v3/projects/d9d5b422d73147ffbcc7e623a15c38a5/users/cfe8512bb42f402c9c61480f809ca759/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:54.630964 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62508d92-fbb9-4f22-9c1d-f08db7812d6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9d5b422d73147ffbcc7e623a15c38a5, user_id=cfe8512bb42f402c9c61480f809ca759, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:54.633678 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62508d92-fbb9-4f22-9c1d-f08db7812d6b admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:54.644611 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1298/2625] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 02:03:54 2022] PUT /identity/v3/projects/d9d5b422d73147ffbcc7e623a15c38a5/users/cfe8512bb42f402c9c61480f809ca759/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:03:54.656403 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76a95d-7bb4-4f55-87da-262134d17af8 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:54.656694 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76a95d-7bb4-4f55-87da-262134d17af8 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:54.656985 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76a95d-7bb4-4f55-87da-262134d17af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:54.692629 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-3e76a95d-7bb4-4f55-87da-262134d17af8 None None] MFA Rules not processed for user `cfe8512bb42f402c9c61480f809ca759`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:03:54.693833 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-3e76a95d-7bb4-4f55-87da-262134d17af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:03:54.727560 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:03:54.764131 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1328/2626] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:03:54 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 02:03:54.775577 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62508d92-fbb9-4f22-9c1d-f08db7812d6b admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:54.777648 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-62508d92-fbb9-4f22-9c1d-f08db7812d6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:03:54.838036 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62508d92-fbb9-4f22-9c1d-f08db7812d6b admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:54.842070 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f6d1cf-0108-4e71-be9c-606396403044 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:54.842758 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6d1cf-0108-4e71-be9c-606396403044 service cinder] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:54.843036 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6d1cf-0108-4e71-be9c-606396403044 service cinder] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:54.843339 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6d1cf-0108-4e71-be9c-606396403044 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:54.853044 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f6d1cf-0108-4e71-be9c-606396403044 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:54.854474 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f6d1cf-0108-4e71-be9c-606396403044 service cinder] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:54.864186 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1299/2627] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:03:54 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:03:54.905980 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e76a95d-7bb4-4f55-87da-262134d17af8 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:54.907488 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-3e76a95d-7bb4-4f55-87da-262134d17af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:03:54.919828 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e76a95d-7bb4-4f55-87da-262134d17af8 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:54.923577 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4967bdb7-734c-4123-a3ff-e5a1ae40cf42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:54.924213 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967bdb7-734c-4123-a3ff-e5a1ae40cf42 service cinder] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:54.924494 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967bdb7-734c-4123-a3ff-e5a1ae40cf42 service cinder] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:54.924780 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967bdb7-734c-4123-a3ff-e5a1ae40cf42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:54.982965 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4967bdb7-734c-4123-a3ff-e5a1ae40cf42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:54.984451 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4967bdb7-734c-4123-a3ff-e5a1ae40cf42 service cinder] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:54.993735 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1329/2628] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:03:54 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:03:57.895085 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f6d1cf-0108-4e71-be9c-606396403044 service cinder] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:57.901173 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:03:57.962735 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f6d1cf-0108-4e71-be9c-606396403044 service cinder] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:57.967575 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ded094-3dec-4f97-aa7d-97b36aabafcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:57.968449 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ded094-3dec-4f97-aa7d-97b36aabafcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:57.968729 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ded094-3dec-4f97-aa7d-97b36aabafcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:57.969005 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ded094-3dec-4f97-aa7d-97b36aabafcb admin admin] PATH_INFO: `/v3/users/a9cbd7f2a3ba4894b2059d79a043416a` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:57.970223 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ded094-3dec-4f97-aa7d-97b36aabafcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9cbd7f2a3ba4894b2059d79a043416a)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:57.972186 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ded094-3dec-4f97-aa7d-97b36aabafcb admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:58.028549 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-85ded094-3dec-4f97-aa7d-97b36aabafcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9cbd7f2a3ba4894b2059d79a043416a'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:58.042127 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-85ded094-3dec-4f97-aa7d-97b36aabafcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9cbd7f2a3ba4894b2059d79a043416a'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:58.051645 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-85ded094-3dec-4f97-aa7d-97b36aabafcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9cbd7f2a3ba4894b2059d79a043416a'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:58.058987 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-85ded094-3dec-4f97-aa7d-97b36aabafcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9cbd7f2a3ba4894b2059d79a043416a'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:58.059767 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-85ded094-3dec-4f97-aa7d-97b36aabafcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9cbd7f2a3ba4894b2059d79a043416a'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:58.065985 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1300/2629] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 02:03:57 2022] DELETE /identity/v3/users/a9cbd7f2a3ba4894b2059d79a043416a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:03:58.070579 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b168e9-dbec-4bc1-9f3c-965fc0d948b9 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:58.071137 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b168e9-dbec-4bc1-9f3c-965fc0d948b9 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:58.071137 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b168e9-dbec-4bc1-9f3c-965fc0d948b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:58.088268 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-87b168e9-dbec-4bc1-9f3c-965fc0d948b9 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:03:58.089451 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-87b168e9-dbec-4bc1-9f3c-965fc0d948b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:03:58.155040 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1330/2630] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:03:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:03:58.167635 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ded094-3dec-4f97-aa7d-97b36aabafcb admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:58.169172 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-85ded094-3dec-4f97-aa7d-97b36aabafcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:03:58.237500 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ded094-3dec-4f97-aa7d-97b36aabafcb admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:58.241816 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0a144-16d4-4a6a-b15b-554d7a5d374f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:58.242454 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0a144-16d4-4a6a-b15b-554d7a5d374f service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:58.242745 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0a144-16d4-4a6a-b15b-554d7a5d374f service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:58.243019 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0a144-16d4-4a6a-b15b-554d7a5d374f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:58.253460 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f0a144-16d4-4a6a-b15b-554d7a5d374f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:58.254902 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f0a144-16d4-4a6a-b15b-554d7a5d374f service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:58.265169 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1301/2631] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:03:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:03:58.541257 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b168e9-dbec-4bc1-9f3c-965fc0d948b9 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:58.542699 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-87b168e9-dbec-4bc1-9f3c-965fc0d948b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:03:58.555481 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b168e9-dbec-4bc1-9f3c-965fc0d948b9 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:58.561223 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499c6592-6e6a-4986-b622-93d943670b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:58.562170 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c6592-6e6a-4986-b622-93d943670b63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:58.562530 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c6592-6e6a-4986-b622-93d943670b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:58.562851 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c6592-6e6a-4986-b622-93d943670b63 admin admin] PATH_INFO: `/v3/projects/6e828073368644c18d654a07f3e56a52` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:58.564149 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499c6592-6e6a-4986-b622-93d943670b63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e828073368644c18d654a07f3e56a52)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:58.566047 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499c6592-6e6a-4986-b622-93d943670b63 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:58.613688 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-499c6592-6e6a-4986-b622-93d943670b63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e828073368644c18d654a07f3e56a52', 'request_id': 'req-499c6592-6e6a-4986-b622-93d943670b63'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:58.619911 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-499c6592-6e6a-4986-b622-93d943670b63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e828073368644c18d654a07f3e56a52', 'request_id': 'req-499c6592-6e6a-4986-b622-93d943670b63'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:58.628644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-499c6592-6e6a-4986-b622-93d943670b63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e828073368644c18d654a07f3e56a52', 'request_id': 'req-499c6592-6e6a-4986-b622-93d943670b63'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:58.631186 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:03:58.640990 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-499c6592-6e6a-4986-b622-93d943670b63 admin admin] The token cache is being invalidate because project 6e828073368644c18d654a07f3e56a52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 02:03:58.641270 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-499c6592-6e6a-4986-b622-93d943670b63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:58.641875 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-499c6592-6e6a-4986-b622-93d943670b63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:58.643812 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1331/2632] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 02:03:58 2022] DELETE /identity/v3/projects/6e828073368644c18d654a07f3e56a52 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 02:03:58.647521 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0a144-16d4-4a6a-b15b-554d7a5d374f service neutron] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:58.657863 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0a144-16d4-4a6a-b15b-554d7a5d374f service neutron] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:58.684268 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:03:58.720988 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c185fd58-b57f-498b-83e8-c2e11234d751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:58.721982 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c185fd58-b57f-498b-83e8-c2e11234d751 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:58.722267 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c185fd58-b57f-498b-83e8-c2e11234d751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:58.722537 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c185fd58-b57f-498b-83e8-c2e11234d751 admin admin] PATH_INFO: `/v3/users/ce1ab5284203403c8bcffbcf3a26c88f` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:58.723869 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c185fd58-b57f-498b-83e8-c2e11234d751 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce1ab5284203403c8bcffbcf3a26c88f)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:58.725815 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c185fd58-b57f-498b-83e8-c2e11234d751 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:58.774769 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-c185fd58-b57f-498b-83e8-c2e11234d751 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce1ab5284203403c8bcffbcf3a26c88f'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:58.786631 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-c185fd58-b57f-498b-83e8-c2e11234d751 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce1ab5284203403c8bcffbcf3a26c88f'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:58.795810 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-c185fd58-b57f-498b-83e8-c2e11234d751 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce1ab5284203403c8bcffbcf3a26c88f'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:58.803649 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-c185fd58-b57f-498b-83e8-c2e11234d751 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce1ab5284203403c8bcffbcf3a26c88f'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:58.804331 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-c185fd58-b57f-498b-83e8-c2e11234d751 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce1ab5284203403c8bcffbcf3a26c88f'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:58.810475 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1302/2633] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 02:03:58 2022] DELETE /identity/v3/users/ce1ab5284203403c8bcffbcf3a26c88f => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:03:59.086129 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499c6592-6e6a-4986-b622-93d943670b63 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:59.087631 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-499c6592-6e6a-4986-b622-93d943670b63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:03:59.101713 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499c6592-6e6a-4986-b622-93d943670b63 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:59.161778 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef62f09f-67f3-49f5-b1eb-eba7995fc72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:59.162649 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62f09f-67f3-49f5-b1eb-eba7995fc72f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:59.162927 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62f09f-67f3-49f5-b1eb-eba7995fc72f admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:59.163343 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62f09f-67f3-49f5-b1eb-eba7995fc72f admin admin] PATH_INFO: `/v3/projects/6b44be1decb34d81bbe710d73ba6bc50` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:59.164629 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef62f09f-67f3-49f5-b1eb-eba7995fc72f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b44be1decb34d81bbe710d73ba6bc50)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:59.166893 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef62f09f-67f3-49f5-b1eb-eba7995fc72f admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:59.216482 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-ef62f09f-67f3-49f5-b1eb-eba7995fc72f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b44be1decb34d81bbe710d73ba6bc50', 'request_id': 'req-ef62f09f-67f3-49f5-b1eb-eba7995fc72f'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:59.222562 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-ef62f09f-67f3-49f5-b1eb-eba7995fc72f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b44be1decb34d81bbe710d73ba6bc50', 'request_id': 'req-ef62f09f-67f3-49f5-b1eb-eba7995fc72f'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:59.230983 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-ef62f09f-67f3-49f5-b1eb-eba7995fc72f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b44be1decb34d81bbe710d73ba6bc50', 'request_id': 'req-ef62f09f-67f3-49f5-b1eb-eba7995fc72f'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:59.248002 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-ef62f09f-67f3-49f5-b1eb-eba7995fc72f admin admin] The token cache is being invalidate because project 6b44be1decb34d81bbe710d73ba6bc50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 02:03:59.248324 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-ef62f09f-67f3-49f5-b1eb-eba7995fc72f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:59.248977 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-ef62f09f-67f3-49f5-b1eb-eba7995fc72f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:59.250545 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1332/2634] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 02:03:59 2022] DELETE /identity/v3/projects/6b44be1decb34d81bbe710d73ba6bc50 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:03:59.254874 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c185fd58-b57f-498b-83e8-c2e11234d751 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:59.256336 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-c185fd58-b57f-498b-83e8-c2e11234d751 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:03:59.269386 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c185fd58-b57f-498b-83e8-c2e11234d751 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:03:59.340920 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be6732-11d3-43a2-9010-103d7e7a3ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:03:59.341875 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be6732-11d3-43a2-9010-103d7e7a3ed3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:03:59.342152 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be6732-11d3-43a2-9010-103d7e7a3ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:03:59.342505 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be6732-11d3-43a2-9010-103d7e7a3ed3 admin admin] PATH_INFO: `/v3/users/cfe8512bb42f402c9c61480f809ca759` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:03:59.343768 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39be6732-11d3-43a2-9010-103d7e7a3ed3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfe8512bb42f402c9c61480f809ca759)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:03:59.345711 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39be6732-11d3-43a2-9010-103d7e7a3ed3 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:03:59.400545 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-39be6732-11d3-43a2-9010-103d7e7a3ed3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfe8512bb42f402c9c61480f809ca759'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:59.401800 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:03:59.415269 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-39be6732-11d3-43a2-9010-103d7e7a3ed3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfe8512bb42f402c9c61480f809ca759'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:59.423630 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-39be6732-11d3-43a2-9010-103d7e7a3ed3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfe8512bb42f402c9c61480f809ca759'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:59.431038 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-39be6732-11d3-43a2-9010-103d7e7a3ed3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfe8512bb42f402c9c61480f809ca759'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:59.431785 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-39be6732-11d3-43a2-9010-103d7e7a3ed3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfe8512bb42f402c9c61480f809ca759'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:03:59.438004 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1303/2635] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 02:03:59 2022] DELETE /identity/v3/users/cfe8512bb42f402c9c61480f809ca759 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 02:03:59.965655 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef62f09f-67f3-49f5-b1eb-eba7995fc72f admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:03:59.967512 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-ef62f09f-67f3-49f5-b1eb-eba7995fc72f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:03:59.980345 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef62f09f-67f3-49f5-b1eb-eba7995fc72f admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:00.039663 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c61a46b-8bc3-4156-b781-daf4099314d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:00.040817 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61a46b-8bc3-4156-b781-daf4099314d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:00.041101 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61a46b-8bc3-4156-b781-daf4099314d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:00.041377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61a46b-8bc3-4156-b781-daf4099314d6 admin admin] PATH_INFO: `/v3/projects/d9d5b422d73147ffbcc7e623a15c38a5` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:00.042624 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c61a46b-8bc3-4156-b781-daf4099314d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9d5b422d73147ffbcc7e623a15c38a5)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:00.044620 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c61a46b-8bc3-4156-b781-daf4099314d6 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:00.069253 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:04:00.096048 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-0c61a46b-8bc3-4156-b781-daf4099314d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9d5b422d73147ffbcc7e623a15c38a5', 'request_id': 'req-0c61a46b-8bc3-4156-b781-daf4099314d6'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:00.102472 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-0c61a46b-8bc3-4156-b781-daf4099314d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9d5b422d73147ffbcc7e623a15c38a5', 'request_id': 'req-0c61a46b-8bc3-4156-b781-daf4099314d6'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:00.111445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-0c61a46b-8bc3-4156-b781-daf4099314d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9d5b422d73147ffbcc7e623a15c38a5', 'request_id': 'req-0c61a46b-8bc3-4156-b781-daf4099314d6'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:00.121751 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-0c61a46b-8bc3-4156-b781-daf4099314d6 admin admin] The token cache is being invalidate because project d9d5b422d73147ffbcc7e623a15c38a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 02:04:00.122065 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-0c61a46b-8bc3-4156-b781-daf4099314d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:00.122674 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-0c61a46b-8bc3-4156-b781-daf4099314d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:00.124328 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1333/2636] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 02:03:59 2022] DELETE /identity/v3/projects/d9d5b422d73147ffbcc7e623a15c38a5 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:04:00.130375 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6eef2-2491-49a5-b817-6ee138598674 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:00.130658 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6eef2-2491-49a5-b817-6ee138598674 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:00.130932 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6eef2-2491-49a5-b817-6ee138598674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:00.149725 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-92f6eef2-2491-49a5-b817-6ee138598674 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:04:00.150998 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-92f6eef2-2491-49a5-b817-6ee138598674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:00.214349 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1304/2637] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:04:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:04:00.222919 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea5245-53c7-44f9-84a6-1a9754cd9a48 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:00.223234 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea5245-53c7-44f9-84a6-1a9754cd9a48 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:00.223514 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea5245-53c7-44f9-84a6-1a9754cd9a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:00.245245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-33ea5245-53c7-44f9-84a6-1a9754cd9a48 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:04:00.246974 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-33ea5245-53c7-44f9-84a6-1a9754cd9a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:00.257340 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1334/2638] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:04:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:04:00.262186 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f6eef2-2491-49a5-b817-6ee138598674 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:00.273807 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f6eef2-2491-49a5-b817-6ee138598674 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:00.280606 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7696b80-5239-48a6-819b-555663215538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:00.281313 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7696b80-5239-48a6-819b-555663215538 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:00.281596 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7696b80-5239-48a6-819b-555663215538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:00.281873 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7696b80-5239-48a6-819b-555663215538 admin admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:00.282611 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7696b80-5239-48a6-819b-555663215538 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 02:04:00.282918 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7696b80-5239-48a6-819b-555663215538 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:00.284570 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7696b80-5239-48a6-819b-555663215538 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:00.297523 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1305/2639] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 02:04:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 02:04:00.302545 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea5245-53c7-44f9-84a6-1a9754cd9a48 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:00.312284 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea5245-53c7-44f9-84a6-1a9754cd9a48 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:00.317133 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74692096-7f3d-49c6-b0dd-ee735267edc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:00.317782 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74692096-7f3d-49c6-b0dd-ee735267edc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:00.318063 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74692096-7f3d-49c6-b0dd-ee735267edc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:00.318337 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74692096-7f3d-49c6-b0dd-ee735267edc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:00.319027 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74692096-7f3d-49c6-b0dd-ee735267edc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:00.320982 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74692096-7f3d-49c6-b0dd-ee735267edc4 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:00.349034 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1335/2640] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 02:04:00 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:04:00.352593 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7696b80-5239-48a6-819b-555663215538 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:00.362646 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7696b80-5239-48a6-819b-555663215538 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:00.419851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae56dbd0-f5b5-4ea6-bf40-3cb9d53a088a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:00.420596 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae56dbd0-f5b5-4ea6-bf40-3cb9d53a088a admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:00.420876 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae56dbd0-f5b5-4ea6-bf40-3cb9d53a088a admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:00.421151 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae56dbd0-f5b5-4ea6-bf40-3cb9d53a088a admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:00.421830 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae56dbd0-f5b5-4ea6-bf40-3cb9d53a088a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:00.423784 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae56dbd0-f5b5-4ea6-bf40-3cb9d53a088a admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:00.444352 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1306/2641] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 02:04:00 2022] POST /identity/v3/users => generated 284 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:04:00.448486 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74692096-7f3d-49c6-b0dd-ee735267edc4 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:00.459308 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74692096-7f3d-49c6-b0dd-ee735267edc4 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:00.464937 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fa24aa-810f-4e75-bd79-c067b28a2bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:00.465534 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa24aa-810f-4e75-bd79-c067b28a2bdf admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:00.465818 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa24aa-810f-4e75-bd79-c067b28a2bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:00.466086 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa24aa-810f-4e75-bd79-c067b28a2bdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:00.466820 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fa24aa-810f-4e75-bd79-c067b28a2bdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:00.468475 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fa24aa-810f-4e75-bd79-c067b28a2bdf admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:00.479291 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1336/2642] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 02:04:00.483359 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae56dbd0-f5b5-4ea6-bf40-3cb9d53a088a admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:00.493572 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae56dbd0-f5b5-4ea6-bf40-3cb9d53a088a admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:00.498705 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d0e73b-c38c-4677-8071-8fe89850458a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:00.499390 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0e73b-c38c-4677-8071-8fe89850458a admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:00.499672 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0e73b-c38c-4677-8071-8fe89850458a admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:00.499945 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0e73b-c38c-4677-8071-8fe89850458a admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:00.500675 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d0e73b-c38c-4677-8071-8fe89850458a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:00.502297 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d0e73b-c38c-4677-8071-8fe89850458a admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:00.512455 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1307/2643] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:04:00.516848 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fa24aa-810f-4e75-bd79-c067b28a2bdf admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:00.527096 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fa24aa-810f-4e75-bd79-c067b28a2bdf admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:00.532064 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756f3567-e628-407b-b664-d4711e6550ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:00.532933 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756f3567-e628-407b-b664-d4711e6550ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:00.533206 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756f3567-e628-407b-b664-d4711e6550ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:00.533477 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756f3567-e628-407b-b664-d4711e6550ea admin admin] PATH_INFO: `/v3/projects/20eeadafe1e14d92aa9d96622392080d/users/8ffb12b4ceef402aba903f0ff5c0d99e/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:00.549014 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756f3567-e628-407b-b664-d4711e6550ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=20eeadafe1e14d92aa9d96622392080d, user_id=8ffb12b4ceef402aba903f0ff5c0d99e, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:00.551840 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756f3567-e628-407b-b664-d4711e6550ea admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:00.566677 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1337/2644] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 02:04:00 2022] PUT /identity/v3/projects/20eeadafe1e14d92aa9d96622392080d/users/8ffb12b4ceef402aba903f0ff5c0d99e/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:04:00.577954 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f09536-7c01-47fa-aef2-043b309e541d None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:00.578255 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f09536-7c01-47fa-aef2-043b309e541d None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:00.578535 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f09536-7c01-47fa-aef2-043b309e541d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:00.606355 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-42f09536-7c01-47fa-aef2-043b309e541d None None] MFA Rules not processed for user `8ffb12b4ceef402aba903f0ff5c0d99e`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:04:00.607639 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-42f09536-7c01-47fa-aef2-043b309e541d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:00.626128 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:04:00.678641 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1308/2645] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:04:00 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:04:00.683405 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756f3567-e628-407b-b664-d4711e6550ea admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:00.752005 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756f3567-e628-407b-b664-d4711e6550ea admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:00.757000 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd3f72a-409d-4a18-89cb-76ed53ced8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:00.757640 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3f72a-409d-4a18-89cb-76ed53ced8ed admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:00.757917 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3f72a-409d-4a18-89cb-76ed53ced8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:00.758187 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3f72a-409d-4a18-89cb-76ed53ced8ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:00.758863 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd3f72a-409d-4a18-89cb-76ed53ced8ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:00.760782 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd3f72a-409d-4a18-89cb-76ed53ced8ed admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:00.789005 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1338/2646] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 02:04:00 2022] POST /identity/v3/projects => generated 367 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:04:00.794650 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f09536-7c01-47fa-aef2-043b309e541d None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:00.805582 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f09536-7c01-47fa-aef2-043b309e541d None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:00.862827 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94858328-0c06-4054-b29c-f59c3071f715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:00.863580 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94858328-0c06-4054-b29c-f59c3071f715 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:00.863842 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94858328-0c06-4054-b29c-f59c3071f715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:00.864123 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94858328-0c06-4054-b29c-f59c3071f715 admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:00.864814 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94858328-0c06-4054-b29c-f59c3071f715 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:00.866741 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94858328-0c06-4054-b29c-f59c3071f715 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:00.891272 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1309/2647] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 02:04:00 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:04:00.896103 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd3f72a-409d-4a18-89cb-76ed53ced8ed admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:00.909977 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd3f72a-409d-4a18-89cb-76ed53ced8ed admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:00.916110 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f8c606-985a-49b8-ad78-aa4a95c0f0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:00.916762 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f8c606-985a-49b8-ad78-aa4a95c0f0a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:00.917057 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f8c606-985a-49b8-ad78-aa4a95c0f0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:00.917307 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f8c606-985a-49b8-ad78-aa4a95c0f0a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:00.918038 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f8c606-985a-49b8-ad78-aa4a95c0f0a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:00.919715 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f8c606-985a-49b8-ad78-aa4a95c0f0a8 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:00.930374 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1339/2648] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:04:00.934384 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94858328-0c06-4054-b29c-f59c3071f715 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:00.944776 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94858328-0c06-4054-b29c-f59c3071f715 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:00.950647 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e5afd6-b6f1-4343-825a-5404c4dfdcb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:00.951588 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5afd6-b6f1-4343-825a-5404c4dfdcb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:00.951873 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5afd6-b6f1-4343-825a-5404c4dfdcb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:00.952149 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5afd6-b6f1-4343-825a-5404c4dfdcb4 admin admin] PATH_INFO: `/v3/projects/1e86428bc0db49a9adb4bb6f1c333a48/users/bf1a9a33b4e74fef98a8b304764573e5/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:00.967581 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e5afd6-b6f1-4343-825a-5404c4dfdcb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e86428bc0db49a9adb4bb6f1c333a48, user_id=bf1a9a33b4e74fef98a8b304764573e5, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:00.970255 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e5afd6-b6f1-4343-825a-5404c4dfdcb4 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:00.986038 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1310/2649] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 02:04:00 2022] PUT /identity/v3/projects/1e86428bc0db49a9adb4bb6f1c333a48/users/bf1a9a33b4e74fef98a8b304764573e5/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:04:00.990248 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f8c606-985a-49b8-ad78-aa4a95c0f0a8 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:01.001833 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f8c606-985a-49b8-ad78-aa4a95c0f0a8 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:01.021350 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:04:01.062989 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb93c2-1fd3-451c-b820-a97661eba1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:01.063681 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb93c2-1fd3-451c-b820-a97661eba1fe admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:01.063957 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb93c2-1fd3-451c-b820-a97661eba1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:01.064233 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb93c2-1fd3-451c-b820-a97661eba1fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:01.064945 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eb93c2-1fd3-451c-b820-a97661eba1fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:01.066546 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eb93c2-1fd3-451c-b820-a97661eba1fe admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:01.076392 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1340/2650] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:04:01.080918 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e5afd6-b6f1-4343-825a-5404c4dfdcb4 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:01.091618 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e5afd6-b6f1-4343-825a-5404c4dfdcb4 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:01.097326 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2480ed1-350d-4038-a1f5-21902861883e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:01.098220 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2480ed1-350d-4038-a1f5-21902861883e admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:01.098501 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2480ed1-350d-4038-a1f5-21902861883e admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:01.098775 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2480ed1-350d-4038-a1f5-21902861883e admin admin] PATH_INFO: `/v3/projects/1e86428bc0db49a9adb4bb6f1c333a48/users/bf1a9a33b4e74fef98a8b304764573e5/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:01.101019 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2480ed1-350d-4038-a1f5-21902861883e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e86428bc0db49a9adb4bb6f1c333a48, user_id=bf1a9a33b4e74fef98a8b304764573e5, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:01.103785 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2480ed1-350d-4038-a1f5-21902861883e admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:01.115351 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1311/2651] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 02:04:01 2022] PUT /identity/v3/projects/1e86428bc0db49a9adb4bb6f1c333a48/users/bf1a9a33b4e74fef98a8b304764573e5/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:04:01.126379 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d30057-9981-409c-9d34-796df75e39fb None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:01.126875 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d30057-9981-409c-9d34-796df75e39fb None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:01.126992 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d30057-9981-409c-9d34-796df75e39fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:01.156336 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-56d30057-9981-409c-9d34-796df75e39fb None None] MFA Rules not processed for user `bf1a9a33b4e74fef98a8b304764573e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:04:01.157565 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-56d30057-9981-409c-9d34-796df75e39fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:01.233309 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1341/2652] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:04:01 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:04:01.245699 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2480ed1-350d-4038-a1f5-21902861883e admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:01.247518 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-f2480ed1-350d-4038-a1f5-21902861883e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:01.308643 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2480ed1-350d-4038-a1f5-21902861883e admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:01.312703 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6c86c-42cc-4add-88c9-dddff6214791 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:01.313357 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6c86c-42cc-4add-88c9-dddff6214791 service cinder] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:01.313625 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6c86c-42cc-4add-88c9-dddff6214791 service cinder] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:01.313905 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6c86c-42cc-4add-88c9-dddff6214791 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:01.323206 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b6c86c-42cc-4add-88c9-dddff6214791 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:01.324670 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b6c86c-42cc-4add-88c9-dddff6214791 service cinder] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:01.334534 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1312/2653] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:04:01 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:04:01.422541 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d30057-9981-409c-9d34-796df75e39fb None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:01.433332 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d30057-9981-409c-9d34-796df75e39fb None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:01.437408 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ec3a7f-acd5-4bd6-970a-a386d8dcb494 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:01.438022 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ec3a7f-acd5-4bd6-970a-a386d8dcb494 service cinder] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:01.438325 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ec3a7f-acd5-4bd6-970a-a386d8dcb494 service cinder] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:01.438593 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ec3a7f-acd5-4bd6-970a-a386d8dcb494 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:01.502198 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ec3a7f-acd5-4bd6-970a-a386d8dcb494 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:01.503686 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ec3a7f-acd5-4bd6-970a-a386d8dcb494 service cinder] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:01.513150 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1342/2654] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:04:01 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:04:06.253893 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6c86c-42cc-4add-88c9-dddff6214791 service cinder] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:06.275239 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:04:06.323363 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6c86c-42cc-4add-88c9-dddff6214791 service cinder] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:06.328569 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb7358b-09bb-4e24-bd2a-ea742230d2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:06.329531 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7358b-09bb-4e24-bd2a-ea742230d2f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:06.329796 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7358b-09bb-4e24-bd2a-ea742230d2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:06.330068 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7358b-09bb-4e24-bd2a-ea742230d2f7 admin admin] PATH_INFO: `/v3/users/8ffb12b4ceef402aba903f0ff5c0d99e` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:06.331401 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb7358b-09bb-4e24-bd2a-ea742230d2f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ffb12b4ceef402aba903f0ff5c0d99e)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:06.333358 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb7358b-09bb-4e24-bd2a-ea742230d2f7 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:06.392022 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-9fb7358b-09bb-4e24-bd2a-ea742230d2f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ffb12b4ceef402aba903f0ff5c0d99e'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:06.405101 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-9fb7358b-09bb-4e24-bd2a-ea742230d2f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ffb12b4ceef402aba903f0ff5c0d99e'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:06.415732 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-9fb7358b-09bb-4e24-bd2a-ea742230d2f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ffb12b4ceef402aba903f0ff5c0d99e'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:06.423072 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-9fb7358b-09bb-4e24-bd2a-ea742230d2f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ffb12b4ceef402aba903f0ff5c0d99e'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:06.423760 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-9fb7358b-09bb-4e24-bd2a-ea742230d2f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ffb12b4ceef402aba903f0ff5c0d99e'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:06.429778 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1313/2655] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 02:04:06 2022] DELETE /identity/v3/users/8ffb12b4ceef402aba903f0ff5c0d99e => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:04:06.434478 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6404d-8919-4414-af37-943c64bed569 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:06.434982 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6404d-8919-4414-af37-943c64bed569 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:06.434982 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6404d-8919-4414-af37-943c64bed569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:06.453065 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-5ff6404d-8919-4414-af37-943c64bed569 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:04:06.454273 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-5ff6404d-8919-4414-af37-943c64bed569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:06.501359 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:04:06.523232 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1343/2656] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:04:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:04:06.533255 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb7358b-09bb-4e24-bd2a-ea742230d2f7 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:06.534828 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-9fb7358b-09bb-4e24-bd2a-ea742230d2f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:06.602684 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb7358b-09bb-4e24-bd2a-ea742230d2f7 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:06.607575 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b35780-0560-4d3a-9d95-a11bfe49610f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:06.609190 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b35780-0560-4d3a-9d95-a11bfe49610f service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:06.609478 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b35780-0560-4d3a-9d95-a11bfe49610f service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:06.609753 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b35780-0560-4d3a-9d95-a11bfe49610f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:06.620671 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b35780-0560-4d3a-9d95-a11bfe49610f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:06.622109 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b35780-0560-4d3a-9d95-a11bfe49610f service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:06.631464 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1314/2657] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:04:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:04:06.899096 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff6404d-8919-4414-af37-943c64bed569 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:06.900600 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-5ff6404d-8919-4414-af37-943c64bed569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:06.913072 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff6404d-8919-4414-af37-943c64bed569 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:06.919094 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cefa8a-9a9f-47ae-88d0-226d066a216d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:06.920096 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cefa8a-9a9f-47ae-88d0-226d066a216d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:06.920370 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cefa8a-9a9f-47ae-88d0-226d066a216d admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:06.920637 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cefa8a-9a9f-47ae-88d0-226d066a216d admin admin] PATH_INFO: `/v3/projects/20eeadafe1e14d92aa9d96622392080d` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:06.921956 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cefa8a-9a9f-47ae-88d0-226d066a216d admin admin] RBAC: Authorizing `identity:delete_project(project_id=20eeadafe1e14d92aa9d96622392080d)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:06.923958 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cefa8a-9a9f-47ae-88d0-226d066a216d admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:06.977140 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-a9cefa8a-9a9f-47ae-88d0-226d066a216d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20eeadafe1e14d92aa9d96622392080d', 'request_id': 'req-a9cefa8a-9a9f-47ae-88d0-226d066a216d'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:06.983369 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-a9cefa8a-9a9f-47ae-88d0-226d066a216d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20eeadafe1e14d92aa9d96622392080d', 'request_id': 'req-a9cefa8a-9a9f-47ae-88d0-226d066a216d'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:06.992222 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-a9cefa8a-9a9f-47ae-88d0-226d066a216d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20eeadafe1e14d92aa9d96622392080d', 'request_id': 'req-a9cefa8a-9a9f-47ae-88d0-226d066a216d'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:07.002647 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-a9cefa8a-9a9f-47ae-88d0-226d066a216d admin admin] The token cache is being invalidate because project 20eeadafe1e14d92aa9d96622392080d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 02:04:07.002936 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-a9cefa8a-9a9f-47ae-88d0-226d066a216d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:07.003573 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-a9cefa8a-9a9f-47ae-88d0-226d066a216d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:07.005112 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1344/2658] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 02:04:06 2022] DELETE /identity/v3/projects/20eeadafe1e14d92aa9d96622392080d => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:04:07.009150 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b35780-0560-4d3a-9d95-a11bfe49610f service neutron] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:07.019301 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b35780-0560-4d3a-9d95-a11bfe49610f service neutron] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:07.054678 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:04:07.079592 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beb953f-e83f-410f-a113-83dda63a6a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:07.080462 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb953f-e83f-410f-a113-83dda63a6a4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:07.080731 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb953f-e83f-410f-a113-83dda63a6a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:07.081006 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb953f-e83f-410f-a113-83dda63a6a4f admin admin] PATH_INFO: `/v3/users/bf1a9a33b4e74fef98a8b304764573e5` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:07.082214 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beb953f-e83f-410f-a113-83dda63a6a4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf1a9a33b4e74fef98a8b304764573e5)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:07.084177 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beb953f-e83f-410f-a113-83dda63a6a4f admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:07.134560 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-7beb953f-e83f-410f-a113-83dda63a6a4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf1a9a33b4e74fef98a8b304764573e5'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:07.147466 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-7beb953f-e83f-410f-a113-83dda63a6a4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf1a9a33b4e74fef98a8b304764573e5'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:07.155822 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-7beb953f-e83f-410f-a113-83dda63a6a4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf1a9a33b4e74fef98a8b304764573e5'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:07.162669 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-7beb953f-e83f-410f-a113-83dda63a6a4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf1a9a33b4e74fef98a8b304764573e5'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:07.163342 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-7beb953f-e83f-410f-a113-83dda63a6a4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf1a9a33b4e74fef98a8b304764573e5'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:07.169547 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1315/2659] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 02:04:07 2022] DELETE /identity/v3/users/bf1a9a33b4e74fef98a8b304764573e5 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:04:07.469784 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cefa8a-9a9f-47ae-88d0-226d066a216d admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:07.471270 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-a9cefa8a-9a9f-47ae-88d0-226d066a216d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:07.484979 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cefa8a-9a9f-47ae-88d0-226d066a216d admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:07.546031 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d13c23-a161-481b-9f71-e730a4fc2e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:07.546902 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d13c23-a161-481b-9f71-e730a4fc2e43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:07.547181 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d13c23-a161-481b-9f71-e730a4fc2e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:07.547492 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d13c23-a161-481b-9f71-e730a4fc2e43 admin admin] PATH_INFO: `/v3/projects/1e86428bc0db49a9adb4bb6f1c333a48` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:07.548904 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d13c23-a161-481b-9f71-e730a4fc2e43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e86428bc0db49a9adb4bb6f1c333a48)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:07.550851 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d13c23-a161-481b-9f71-e730a4fc2e43 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:07.610817 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-83d13c23-a161-481b-9f71-e730a4fc2e43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e86428bc0db49a9adb4bb6f1c333a48', 'request_id': 'req-83d13c23-a161-481b-9f71-e730a4fc2e43'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:07.620377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-83d13c23-a161-481b-9f71-e730a4fc2e43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e86428bc0db49a9adb4bb6f1c333a48', 'request_id': 'req-83d13c23-a161-481b-9f71-e730a4fc2e43'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:07.631157 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-83d13c23-a161-481b-9f71-e730a4fc2e43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e86428bc0db49a9adb4bb6f1c333a48', 'request_id': 'req-83d13c23-a161-481b-9f71-e730a4fc2e43'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:07.635900 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:04:07.644244 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-83d13c23-a161-481b-9f71-e730a4fc2e43 admin admin] The token cache is being invalidate because project 1e86428bc0db49a9adb4bb6f1c333a48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 02:04:07.644735 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-83d13c23-a161-481b-9f71-e730a4fc2e43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:07.645465 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-83d13c23-a161-481b-9f71-e730a4fc2e43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:07.647117 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1345/2660] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 02:04:07 2022] DELETE /identity/v3/projects/1e86428bc0db49a9adb4bb6f1c333a48 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:04:07.652125 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4038e63-febe-4648-b6b4-f7292aad5628 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:07.652615 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4038e63-febe-4648-b6b4-f7292aad5628 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:07.652730 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4038e63-febe-4648-b6b4-f7292aad5628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:07.671389 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-f4038e63-febe-4648-b6b4-f7292aad5628 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:04:07.672515 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-f4038e63-febe-4648-b6b4-f7292aad5628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:07.746188 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1316/2661] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:04:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 02:04:07.754435 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dce69d-fa22-4252-afdd-d75581b5a783 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:07.754728 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dce69d-fa22-4252-afdd-d75581b5a783 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:07.755005 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dce69d-fa22-4252-afdd-d75581b5a783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:07.774622 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-19dce69d-fa22-4252-afdd-d75581b5a783 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:04:07.775888 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-19dce69d-fa22-4252-afdd-d75581b5a783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:07.788818 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1346/2662] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:04:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:04:07.795263 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4038e63-febe-4648-b6b4-f7292aad5628 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:07.805050 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4038e63-febe-4648-b6b4-f7292aad5628 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:07.810493 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e371fb17-e42f-4743-931e-430b0486b27e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:07.811147 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371fb17-e42f-4743-931e-430b0486b27e admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:07.811459 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371fb17-e42f-4743-931e-430b0486b27e admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:07.811732 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371fb17-e42f-4743-931e-430b0486b27e admin admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:07.812466 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e371fb17-e42f-4743-931e-430b0486b27e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 02:04:07.812774 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e371fb17-e42f-4743-931e-430b0486b27e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:07.814392 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e371fb17-e42f-4743-931e-430b0486b27e admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:07.826905 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1317/2663] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 02:04:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 02:04:07.831272 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dce69d-fa22-4252-afdd-d75581b5a783 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:07.842354 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dce69d-fa22-4252-afdd-d75581b5a783 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:07.851192 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c2267d-fd5b-4864-a2c4-6d558c308fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:07.852156 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c2267d-fd5b-4864-a2c4-6d558c308fbe admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:07.852487 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c2267d-fd5b-4864-a2c4-6d558c308fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:07.852808 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c2267d-fd5b-4864-a2c4-6d558c308fbe admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:07.853869 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c2267d-fd5b-4864-a2c4-6d558c308fbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:07.856822 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c2267d-fd5b-4864-a2c4-6d558c308fbe admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:07.885406 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1347/2664] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 02:04:07 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:04:07.889786 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e371fb17-e42f-4743-931e-430b0486b27e admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:07.900307 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e371fb17-e42f-4743-931e-430b0486b27e admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:07.943438 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:04:07.960265 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345c7fa2-1774-4621-9a7e-b1beb00d8585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:07.960931 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c7fa2-1774-4621-9a7e-b1beb00d8585 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:07.961209 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c7fa2-1774-4621-9a7e-b1beb00d8585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:07.961481 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c7fa2-1774-4621-9a7e-b1beb00d8585 admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:07.962169 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345c7fa2-1774-4621-9a7e-b1beb00d8585 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:07.964135 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345c7fa2-1774-4621-9a7e-b1beb00d8585 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:07.986129 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1318/2665] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 02:04:07 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:04:07.990906 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c2267d-fd5b-4864-a2c4-6d558c308fbe admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:08.006546 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c2267d-fd5b-4864-a2c4-6d558c308fbe admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:08.013753 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6391eb52-ddd9-49f3-8e28-0a80ae2646c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:08.014516 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6391eb52-ddd9-49f3-8e28-0a80ae2646c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:08.014794 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6391eb52-ddd9-49f3-8e28-0a80ae2646c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:08.015096 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6391eb52-ddd9-49f3-8e28-0a80ae2646c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:08.015850 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6391eb52-ddd9-49f3-8e28-0a80ae2646c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:08.017496 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6391eb52-ddd9-49f3-8e28-0a80ae2646c7 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:08.029299 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1348/2666] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:04:08.033514 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345c7fa2-1774-4621-9a7e-b1beb00d8585 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:08.044139 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345c7fa2-1774-4621-9a7e-b1beb00d8585 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:08.049925 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c8be03-14c2-49a1-be26-f1872f8d6155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:08.050590 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c8be03-14c2-49a1-be26-f1872f8d6155 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:08.050872 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c8be03-14c2-49a1-be26-f1872f8d6155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:08.051178 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c8be03-14c2-49a1-be26-f1872f8d6155 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:08.051952 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c8be03-14c2-49a1-be26-f1872f8d6155 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:08.053552 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c8be03-14c2-49a1-be26-f1872f8d6155 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:08.063688 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1319/2667] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:04:08.067917 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6391eb52-ddd9-49f3-8e28-0a80ae2646c7 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:08.077938 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6391eb52-ddd9-49f3-8e28-0a80ae2646c7 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:08.083455 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d41ff23-fb2b-44bc-a1b5-cc5165dc2dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:08.084341 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d41ff23-fb2b-44bc-a1b5-cc5165dc2dc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:08.084625 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d41ff23-fb2b-44bc-a1b5-cc5165dc2dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:08.084903 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d41ff23-fb2b-44bc-a1b5-cc5165dc2dc1 admin admin] PATH_INFO: `/v3/projects/2d34975f39624f84878454edd6eec2e6/users/5c1784c7b96d4e6b83996a692a654eae/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:08.100718 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d41ff23-fb2b-44bc-a1b5-cc5165dc2dc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d34975f39624f84878454edd6eec2e6, user_id=5c1784c7b96d4e6b83996a692a654eae, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:08.103467 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d41ff23-fb2b-44bc-a1b5-cc5165dc2dc1 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:08.119014 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1349/2668] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 02:04:08 2022] PUT /identity/v3/projects/2d34975f39624f84878454edd6eec2e6/users/5c1784c7b96d4e6b83996a692a654eae/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 02:04:08.129306 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5588de-5e4d-457e-b0b5-e22c84de81d0 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:08.129661 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5588de-5e4d-457e-b0b5-e22c84de81d0 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:08.129944 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5588de-5e4d-457e-b0b5-e22c84de81d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:08.160509 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-ec5588de-5e4d-457e-b0b5-e22c84de81d0 None None] MFA Rules not processed for user `5c1784c7b96d4e6b83996a692a654eae`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:04:08.161724 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-ec5588de-5e4d-457e-b0b5-e22c84de81d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:08.228791 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1320/2669] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:04:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 02:04:08.233439 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d41ff23-fb2b-44bc-a1b5-cc5165dc2dc1 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:08.305315 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d41ff23-fb2b-44bc-a1b5-cc5165dc2dc1 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:08.311085 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66366b43-b5dc-4dce-9d99-a67edde9134f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:08.311781 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66366b43-b5dc-4dce-9d99-a67edde9134f admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:08.312065 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66366b43-b5dc-4dce-9d99-a67edde9134f admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:08.312336 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66366b43-b5dc-4dce-9d99-a67edde9134f admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:08.313042 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66366b43-b5dc-4dce-9d99-a67edde9134f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:08.314965 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66366b43-b5dc-4dce-9d99-a67edde9134f admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:08.343514 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1350/2670] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 02:04:08 2022] POST /identity/v3/projects => generated 383 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 02:04:08.347821 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5588de-5e4d-457e-b0b5-e22c84de81d0 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:08.359169 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5588de-5e4d-457e-b0b5-e22c84de81d0 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:08.414738 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd915f59-82f8-4f39-a04e-074316164d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:08.415776 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd915f59-82f8-4f39-a04e-074316164d3b admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:08.416056 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd915f59-82f8-4f39-a04e-074316164d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:08.416324 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd915f59-82f8-4f39-a04e-074316164d3b admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:08.417002 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd915f59-82f8-4f39-a04e-074316164d3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:08.418931 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd915f59-82f8-4f39-a04e-074316164d3b admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:08.439967 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1321/2671] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 02:04:08 2022] POST /identity/v3/users => generated 298 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:04:08.444056 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66366b43-b5dc-4dce-9d99-a67edde9134f admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:08.455007 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66366b43-b5dc-4dce-9d99-a67edde9134f admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:08.459990 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2440ed6-8ad1-4824-b8e5-c8712547c6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:08.460649 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2440ed6-8ad1-4824-b8e5-c8712547c6fb admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:08.460933 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2440ed6-8ad1-4824-b8e5-c8712547c6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:08.461211 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2440ed6-8ad1-4824-b8e5-c8712547c6fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:08.461950 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2440ed6-8ad1-4824-b8e5-c8712547c6fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:08.463598 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2440ed6-8ad1-4824-b8e5-c8712547c6fb admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:08.474485 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1351/2672] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:04:08.478590 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd915f59-82f8-4f39-a04e-074316164d3b admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:08.489218 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd915f59-82f8-4f39-a04e-074316164d3b admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:08.494945 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15487ae6-02c2-4829-846e-da4e080b2c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:08.495840 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15487ae6-02c2-4829-846e-da4e080b2c55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:08.496125 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15487ae6-02c2-4829-846e-da4e080b2c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:08.496400 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15487ae6-02c2-4829-846e-da4e080b2c55 admin admin] PATH_INFO: `/v3/projects/1219adf05ae34d8e8dad3018c73f23a6/users/fe6a550bade14d2ea7b2dbca152a5807/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:08.511916 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15487ae6-02c2-4829-846e-da4e080b2c55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1219adf05ae34d8e8dad3018c73f23a6, user_id=fe6a550bade14d2ea7b2dbca152a5807, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:08.514580 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15487ae6-02c2-4829-846e-da4e080b2c55 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:08.529885 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1322/2673] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 02:04:08 2022] PUT /identity/v3/projects/1219adf05ae34d8e8dad3018c73f23a6/users/fe6a550bade14d2ea7b2dbca152a5807/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:04:08.533989 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2440ed6-8ad1-4824-b8e5-c8712547c6fb admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:08.544425 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2440ed6-8ad1-4824-b8e5-c8712547c6fb admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:08.603886 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d212515d-75d3-4f87-82c0-390e28af6bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:08.605023 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d212515d-75d3-4f87-82c0-390e28af6bf7 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:08.605474 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d212515d-75d3-4f87-82c0-390e28af6bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:08.605944 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d212515d-75d3-4f87-82c0-390e28af6bf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:08.607090 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d212515d-75d3-4f87-82c0-390e28af6bf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:08.609853 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d212515d-75d3-4f87-82c0-390e28af6bf7 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:08.621185 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1352/2674] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:04:08.625572 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15487ae6-02c2-4829-846e-da4e080b2c55 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:08.636848 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15487ae6-02c2-4829-846e-da4e080b2c55 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:08.643311 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1e4ffa-ab38-4a66-80b2-a7097d394f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:08.644178 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1e4ffa-ab38-4a66-80b2-a7097d394f36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:08.644452 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1e4ffa-ab38-4a66-80b2-a7097d394f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:08.644718 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1e4ffa-ab38-4a66-80b2-a7097d394f36 admin admin] PATH_INFO: `/v3/projects/1219adf05ae34d8e8dad3018c73f23a6/users/fe6a550bade14d2ea7b2dbca152a5807/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:08.646867 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1e4ffa-ab38-4a66-80b2-a7097d394f36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1219adf05ae34d8e8dad3018c73f23a6, user_id=fe6a550bade14d2ea7b2dbca152a5807, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:08.649531 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1e4ffa-ab38-4a66-80b2-a7097d394f36 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:08.660278 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1323/2675] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 02:04:08 2022] PUT /identity/v3/projects/1219adf05ae34d8e8dad3018c73f23a6/users/fe6a550bade14d2ea7b2dbca152a5807/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:04:08.675306 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ac059-7a36-46f7-b388-29848035601f None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:08.675593 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ac059-7a36-46f7-b388-29848035601f None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:08.675871 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ac059-7a36-46f7-b388-29848035601f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:08.681818 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:04:08.704613 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-fc4ac059-7a36-46f7-b388-29848035601f None None] MFA Rules not processed for user `fe6a550bade14d2ea7b2dbca152a5807`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:04:08.705828 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-fc4ac059-7a36-46f7-b388-29848035601f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:08.782934 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1353/2676] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:04:08 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:04:08.794267 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1e4ffa-ab38-4a66-80b2-a7097d394f36 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:08.796304 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-7f1e4ffa-ab38-4a66-80b2-a7097d394f36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:08.803257 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:04:08.864137 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1e4ffa-ab38-4a66-80b2-a7097d394f36 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:08.870951 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9119ad-f731-4cdb-9d90-ed04737e5405 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:08.871867 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9119ad-f731-4cdb-9d90-ed04737e5405 service cinder] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:08.872207 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9119ad-f731-4cdb-9d90-ed04737e5405 service cinder] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:08.872532 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9119ad-f731-4cdb-9d90-ed04737e5405 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:08.938889 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9119ad-f731-4cdb-9d90-ed04737e5405 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:08.940357 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9119ad-f731-4cdb-9d90-ed04737e5405 service cinder] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:08.949370 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1324/2677] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:04:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:04:13.130079 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ac059-7a36-46f7-b388-29848035601f None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:13.141069 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ac059-7a36-46f7-b388-29848035601f None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:13.145628 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0af106-0846-4e7a-9ce7-e8fdd19f278c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:13.146273 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0af106-0846-4e7a-9ce7-e8fdd19f278c service cinder] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:13.146550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0af106-0846-4e7a-9ce7-e8fdd19f278c service cinder] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:13.146812 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0af106-0846-4e7a-9ce7-e8fdd19f278c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:13.159975 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0af106-0846-4e7a-9ce7-e8fdd19f278c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:13.161439 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0af106-0846-4e7a-9ce7-e8fdd19f278c service cinder] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:13.171996 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1354/2678] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:04:13 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 02:04:18.080385 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9119ad-f731-4cdb-9d90-ed04737e5405 service cinder] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:18.146734 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9119ad-f731-4cdb-9d90-ed04737e5405 service cinder] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:18.152204 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1f8507-4a1d-4ea7-b202-603c0c7aa858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:18.153108 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f8507-4a1d-4ea7-b202-603c0c7aa858 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:18.153384 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f8507-4a1d-4ea7-b202-603c0c7aa858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:18.153651 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f8507-4a1d-4ea7-b202-603c0c7aa858 admin admin] PATH_INFO: `/v3/users/5c1784c7b96d4e6b83996a692a654eae` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:18.154939 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1f8507-4a1d-4ea7-b202-603c0c7aa858 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c1784c7b96d4e6b83996a692a654eae)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:18.156892 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1f8507-4a1d-4ea7-b202-603c0c7aa858 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:18.213855 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-9f1f8507-4a1d-4ea7-b202-603c0c7aa858 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c1784c7b96d4e6b83996a692a654eae'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:18.226265 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:04:18.228960 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-9f1f8507-4a1d-4ea7-b202-603c0c7aa858 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c1784c7b96d4e6b83996a692a654eae'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:18.237860 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-9f1f8507-4a1d-4ea7-b202-603c0c7aa858 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c1784c7b96d4e6b83996a692a654eae'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:18.244772 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-9f1f8507-4a1d-4ea7-b202-603c0c7aa858 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c1784c7b96d4e6b83996a692a654eae'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:18.245410 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-9f1f8507-4a1d-4ea7-b202-603c0c7aa858 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c1784c7b96d4e6b83996a692a654eae'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:18.251625 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1325/2679] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 02:04:18 2022] DELETE /identity/v3/users/5c1784c7b96d4e6b83996a692a654eae => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:04:18.256240 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa4e33-e5dc-4127-a681-3a79d58dc77a None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:18.256768 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa4e33-e5dc-4127-a681-3a79d58dc77a None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:18.256902 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa4e33-e5dc-4127-a681-3a79d58dc77a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:18.277282 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-feaa4e33-e5dc-4127-a681-3a79d58dc77a None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:04:18.278554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-feaa4e33-e5dc-4127-a681-3a79d58dc77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:18.343076 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1355/2680] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:04:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:04:18.353183 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1f8507-4a1d-4ea7-b202-603c0c7aa858 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:18.354787 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-9f1f8507-4a1d-4ea7-b202-603c0c7aa858 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:18.414200 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1f8507-4a1d-4ea7-b202-603c0c7aa858 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:18.419243 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991b63e-5b80-46dc-bb2b-bc710fe6f74a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:18.419978 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991b63e-5b80-46dc-bb2b-bc710fe6f74a service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:18.420264 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991b63e-5b80-46dc-bb2b-bc710fe6f74a service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:18.420541 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991b63e-5b80-46dc-bb2b-bc710fe6f74a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:18.430175 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3991b63e-5b80-46dc-bb2b-bc710fe6f74a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:18.431695 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3991b63e-5b80-46dc-bb2b-bc710fe6f74a service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:18.441333 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1326/2681] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:04:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:04:18.731915 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaa4e33-e5dc-4127-a681-3a79d58dc77a None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:18.733847 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-feaa4e33-e5dc-4127-a681-3a79d58dc77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:18.748895 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaa4e33-e5dc-4127-a681-3a79d58dc77a None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:18.755510 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe3b39a-c9b0-42a4-8744-adf7d234c8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:18.756510 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe3b39a-c9b0-42a4-8744-adf7d234c8a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:18.756649 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe3b39a-c9b0-42a4-8744-adf7d234c8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:18.757003 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe3b39a-c9b0-42a4-8744-adf7d234c8a5 admin admin] PATH_INFO: `/v3/projects/2d34975f39624f84878454edd6eec2e6` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:18.758297 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe3b39a-c9b0-42a4-8744-adf7d234c8a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d34975f39624f84878454edd6eec2e6)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:18.760303 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe3b39a-c9b0-42a4-8744-adf7d234c8a5 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:18.823307 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-ffe3b39a-c9b0-42a4-8744-adf7d234c8a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d34975f39624f84878454edd6eec2e6', 'request_id': 'req-ffe3b39a-c9b0-42a4-8744-adf7d234c8a5'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:18.830919 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-ffe3b39a-c9b0-42a4-8744-adf7d234c8a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d34975f39624f84878454edd6eec2e6', 'request_id': 'req-ffe3b39a-c9b0-42a4-8744-adf7d234c8a5'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:18.839893 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-ffe3b39a-c9b0-42a4-8744-adf7d234c8a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d34975f39624f84878454edd6eec2e6', 'request_id': 'req-ffe3b39a-c9b0-42a4-8744-adf7d234c8a5'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:18.846316 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:04:18.851882 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-ffe3b39a-c9b0-42a4-8744-adf7d234c8a5 admin admin] The token cache is being invalidate because project 2d34975f39624f84878454edd6eec2e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 02:04:18.852175 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-ffe3b39a-c9b0-42a4-8744-adf7d234c8a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:18.853144 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-ffe3b39a-c9b0-42a4-8744-adf7d234c8a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:18.855051 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1356/2682] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 02:04:18 2022] DELETE /identity/v3/projects/2d34975f39624f84878454edd6eec2e6 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:04:18.859820 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991b63e-5b80-46dc-bb2b-bc710fe6f74a service neutron] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:18.869869 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991b63e-5b80-46dc-bb2b-bc710fe6f74a service neutron] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:18.930972 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a074eab6-0c64-4d82-a1df-a002727bfd7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:18.931906 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a074eab6-0c64-4d82-a1df-a002727bfd7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:18.932187 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a074eab6-0c64-4d82-a1df-a002727bfd7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:18.932510 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a074eab6-0c64-4d82-a1df-a002727bfd7b admin admin] PATH_INFO: `/v3/users/fe6a550bade14d2ea7b2dbca152a5807` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:18.933701 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a074eab6-0c64-4d82-a1df-a002727bfd7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe6a550bade14d2ea7b2dbca152a5807)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:18.935780 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a074eab6-0c64-4d82-a1df-a002727bfd7b admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:18.981179 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-a074eab6-0c64-4d82-a1df-a002727bfd7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe6a550bade14d2ea7b2dbca152a5807'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:18.994650 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-a074eab6-0c64-4d82-a1df-a002727bfd7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe6a550bade14d2ea7b2dbca152a5807'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:19.002783 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-a074eab6-0c64-4d82-a1df-a002727bfd7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe6a550bade14d2ea7b2dbca152a5807'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:19.010081 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-a074eab6-0c64-4d82-a1df-a002727bfd7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe6a550bade14d2ea7b2dbca152a5807'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:19.010846 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-a074eab6-0c64-4d82-a1df-a002727bfd7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe6a550bade14d2ea7b2dbca152a5807'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:19.013856 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:04:19.019580 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1327/2683] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 02:04:18 2022] DELETE /identity/v3/users/fe6a550bade14d2ea7b2dbca152a5807 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:04:19.308091 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe3b39a-c9b0-42a4-8744-adf7d234c8a5 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:19.309753 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-ffe3b39a-c9b0-42a4-8744-adf7d234c8a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:19.337056 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe3b39a-c9b0-42a4-8744-adf7d234c8a5 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:19.405196 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e232c-658a-471d-a0f5-9a9c7de28581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:19.406246 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e232c-658a-471d-a0f5-9a9c7de28581 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:19.406679 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e232c-658a-471d-a0f5-9a9c7de28581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:19.406955 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e232c-658a-471d-a0f5-9a9c7de28581 admin admin] PATH_INFO: `/v3/projects/1219adf05ae34d8e8dad3018c73f23a6` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:19.408380 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36e232c-658a-471d-a0f5-9a9c7de28581 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1219adf05ae34d8e8dad3018c73f23a6)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:19.410331 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36e232c-658a-471d-a0f5-9a9c7de28581 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:19.458885 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-a36e232c-658a-471d-a0f5-9a9c7de28581 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1219adf05ae34d8e8dad3018c73f23a6', 'request_id': 'req-a36e232c-658a-471d-a0f5-9a9c7de28581'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:19.465744 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-a36e232c-658a-471d-a0f5-9a9c7de28581 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1219adf05ae34d8e8dad3018c73f23a6', 'request_id': 'req-a36e232c-658a-471d-a0f5-9a9c7de28581'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:19.475995 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-a36e232c-658a-471d-a0f5-9a9c7de28581 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1219adf05ae34d8e8dad3018c73f23a6', 'request_id': 'req-a36e232c-658a-471d-a0f5-9a9c7de28581'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:19.489689 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-a36e232c-658a-471d-a0f5-9a9c7de28581 admin admin] The token cache is being invalidate because project 1219adf05ae34d8e8dad3018c73f23a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 02:04:19.489980 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-a36e232c-658a-471d-a0f5-9a9c7de28581 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:19.490694 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-a36e232c-658a-471d-a0f5-9a9c7de28581 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:19.492277 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1357/2684] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 02:04:19 2022] DELETE /identity/v3/projects/1219adf05ae34d8e8dad3018c73f23a6 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:04:19.498391 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f455d61-6caf-485f-a96e-2d398cfb4163 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:19.498935 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f455d61-6caf-485f-a96e-2d398cfb4163 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:19.499190 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f455d61-6caf-485f-a96e-2d398cfb4163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:19.522169 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-7f455d61-6caf-485f-a96e-2d398cfb4163 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:04:19.523424 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-7f455d61-6caf-485f-a96e-2d398cfb4163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:19.586634 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1328/2685] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:04:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:04:19.595395 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1498ed36-4b11-4266-8fd2-d245863af831 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:19.595665 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1498ed36-4b11-4266-8fd2-d245863af831 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:19.596058 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1498ed36-4b11-4266-8fd2-d245863af831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:19.617297 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-1498ed36-4b11-4266-8fd2-d245863af831 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:04:19.618823 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-1498ed36-4b11-4266-8fd2-d245863af831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:19.627818 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1358/2686] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:04:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:04:19.632118 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f455d61-6caf-485f-a96e-2d398cfb4163 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:19.643431 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f455d61-6caf-485f-a96e-2d398cfb4163 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:19.648929 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c034b4e-14e3-473e-8aa6-6055b4fa219d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:19.649611 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c034b4e-14e3-473e-8aa6-6055b4fa219d admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:19.649895 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c034b4e-14e3-473e-8aa6-6055b4fa219d admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:19.650157 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c034b4e-14e3-473e-8aa6-6055b4fa219d admin admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:19.650921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c034b4e-14e3-473e-8aa6-6055b4fa219d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 02:04:19.651231 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c034b4e-14e3-473e-8aa6-6055b4fa219d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:19.652975 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c034b4e-14e3-473e-8aa6-6055b4fa219d admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:19.667247 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1329/2687] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 02:04:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 02:04:19.671065 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1498ed36-4b11-4266-8fd2-d245863af831 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:19.680871 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1498ed36-4b11-4266-8fd2-d245863af831 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:19.685687 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7ffd8-6c36-4929-9d25-9521d4ba0dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:19.686412 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7ffd8-6c36-4929-9d25-9521d4ba0dcb admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:19.686765 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7ffd8-6c36-4929-9d25-9521d4ba0dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:19.687104 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7ffd8-6c36-4929-9d25-9521d4ba0dcb admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:19.687908 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c7ffd8-6c36-4929-9d25-9521d4ba0dcb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:19.689885 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c7ffd8-6c36-4929-9d25-9521d4ba0dcb admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:19.721100 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1359/2688] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 02:04:19 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:04:19.722627 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c034b4e-14e3-473e-8aa6-6055b4fa219d admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:19.733002 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c034b4e-14e3-473e-8aa6-6055b4fa219d admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:19.790262 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e251ee-2983-413e-9664-a2d083e0fb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:19.790957 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e251ee-2983-413e-9664-a2d083e0fb61 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:19.791266 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e251ee-2983-413e-9664-a2d083e0fb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:19.791547 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e251ee-2983-413e-9664-a2d083e0fb61 admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:19.792274 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e251ee-2983-413e-9664-a2d083e0fb61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:19.794236 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e251ee-2983-413e-9664-a2d083e0fb61 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:19.819769 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1330/2689] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 02:04:19 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:04:19.824149 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7ffd8-6c36-4929-9d25-9521d4ba0dcb admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:19.835142 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7ffd8-6c36-4929-9d25-9521d4ba0dcb admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:19.840103 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6d03f-d0f6-410a-9000-20c52511e4ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:19.840761 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6d03f-d0f6-410a-9000-20c52511e4ff admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:19.841037 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6d03f-d0f6-410a-9000-20c52511e4ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:19.841305 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6d03f-d0f6-410a-9000-20c52511e4ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:19.842069 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a6d03f-d0f6-410a-9000-20c52511e4ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:19.843773 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a6d03f-d0f6-410a-9000-20c52511e4ff admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:19.854372 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1360/2690] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:04:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 02:04:19.858522 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e251ee-2983-413e-9664-a2d083e0fb61 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:19.868921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e251ee-2983-413e-9664-a2d083e0fb61 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:19.873871 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686063f8-4f30-46a1-959a-875a7e3d0b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:19.874542 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686063f8-4f30-46a1-959a-875a7e3d0b3e admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:19.874853 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686063f8-4f30-46a1-959a-875a7e3d0b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:19.875126 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686063f8-4f30-46a1-959a-875a7e3d0b3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:19.876065 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686063f8-4f30-46a1-959a-875a7e3d0b3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:19.877506 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686063f8-4f30-46a1-959a-875a7e3d0b3e admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:19.887943 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1331/2691] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:04:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:04:19.892274 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6d03f-d0f6-410a-9000-20c52511e4ff admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:19.902446 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6d03f-d0f6-410a-9000-20c52511e4ff admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:19.907446 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3c464d-e5da-4c32-b48b-2b5a6604d1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:19.908680 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c464d-e5da-4c32-b48b-2b5a6604d1a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:19.909584 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c464d-e5da-4c32-b48b-2b5a6604d1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:19.909866 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c464d-e5da-4c32-b48b-2b5a6604d1a9 admin admin] PATH_INFO: `/v3/projects/7a34f672225d4cf7af784445aec2d6a9/users/85258a7ca44347a4a83f7a2cbbcbe3fd/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:19.926225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3c464d-e5da-4c32-b48b-2b5a6604d1a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a34f672225d4cf7af784445aec2d6a9, user_id=85258a7ca44347a4a83f7a2cbbcbe3fd, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:19.928963 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3c464d-e5da-4c32-b48b-2b5a6604d1a9 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:19.943655 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1361/2692] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 02:04:19 2022] PUT /identity/v3/projects/7a34f672225d4cf7af784445aec2d6a9/users/85258a7ca44347a4a83f7a2cbbcbe3fd/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:04:19.956061 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a6f7c-70e1-42ad-806f-10742c2ebbba None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:19.956380 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a6f7c-70e1-42ad-806f-10742c2ebbba None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:19.956656 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a6f7c-70e1-42ad-806f-10742c2ebbba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:19.983756 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-104a6f7c-70e1-42ad-806f-10742c2ebbba None None] MFA Rules not processed for user `85258a7ca44347a4a83f7a2cbbcbe3fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:04:19.985200 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-104a6f7c-70e1-42ad-806f-10742c2ebbba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:20.042305 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:04:20.053014 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1332/2693] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:04:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:04:20.064227 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3c464d-e5da-4c32-b48b-2b5a6604d1a9 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:20.065935 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-ca3c464d-e5da-4c32-b48b-2b5a6604d1a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:20.095272 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:04:20.130358 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3c464d-e5da-4c32-b48b-2b5a6604d1a9 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:20.134893 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d079fb3-7369-4366-b8a8-b2f639ac4c05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:20.135613 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d079fb3-7369-4366-b8a8-b2f639ac4c05 service cinder] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:20.135898 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d079fb3-7369-4366-b8a8-b2f639ac4c05 service cinder] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:20.136155 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d079fb3-7369-4366-b8a8-b2f639ac4c05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:20.146114 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d079fb3-7369-4366-b8a8-b2f639ac4c05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:20.147741 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d079fb3-7369-4366-b8a8-b2f639ac4c05 service cinder] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:20.158667 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1362/2694] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:04:20 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:04:20.179126 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a6f7c-70e1-42ad-806f-10742c2ebbba None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:20.244013 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a6f7c-70e1-42ad-806f-10742c2ebbba None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:20.248998 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b7d37-6f42-4cf2-b4c2-a3e37ad519ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:20.249868 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b7d37-6f42-4cf2-b4c2-a3e37ad519ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:20.250160 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b7d37-6f42-4cf2-b4c2-a3e37ad519ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:20.250438 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b7d37-6f42-4cf2-b4c2-a3e37ad519ca admin admin] PATH_INFO: `/v3/users/85258a7ca44347a4a83f7a2cbbcbe3fd` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:20.251711 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b7d37-6f42-4cf2-b4c2-a3e37ad519ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=85258a7ca44347a4a83f7a2cbbcbe3fd)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:20.253796 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b7d37-6f42-4cf2-b4c2-a3e37ad519ca admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:20.311664 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-2d8b7d37-6f42-4cf2-b4c2-a3e37ad519ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85258a7ca44347a4a83f7a2cbbcbe3fd'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:20.328703 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-2d8b7d37-6f42-4cf2-b4c2-a3e37ad519ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85258a7ca44347a4a83f7a2cbbcbe3fd'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:20.338401 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-2d8b7d37-6f42-4cf2-b4c2-a3e37ad519ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85258a7ca44347a4a83f7a2cbbcbe3fd'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:20.346092 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-2d8b7d37-6f42-4cf2-b4c2-a3e37ad519ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85258a7ca44347a4a83f7a2cbbcbe3fd'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:20.346781 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-2d8b7d37-6f42-4cf2-b4c2-a3e37ad519ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85258a7ca44347a4a83f7a2cbbcbe3fd'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:20.353187 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1333/2695] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 02:04:20 2022] DELETE /identity/v3/users/85258a7ca44347a4a83f7a2cbbcbe3fd => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:04:20.357591 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1013291-dae4-4ca0-9314-477f94988299 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:20.358140 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1013291-dae4-4ca0-9314-477f94988299 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:20.358259 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1013291-dae4-4ca0-9314-477f94988299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:20.376343 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-b1013291-dae4-4ca0-9314-477f94988299 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:04:20.377571 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-b1013291-dae4-4ca0-9314-477f94988299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:20.441727 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1363/2696] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:04:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:04:20.452856 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b7d37-6f42-4cf2-b4c2-a3e37ad519ca admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:20.454452 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-2d8b7d37-6f42-4cf2-b4c2-a3e37ad519ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:20.476238 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c16a7-c1dd-4cac-9d9f-2e30a11cb409 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:20.476526 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c16a7-c1dd-4cac-9d9f-2e30a11cb409 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:20.476807 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c16a7-c1dd-4cac-9d9f-2e30a11cb409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:20.494230 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-240c16a7-c1dd-4cac-9d9f-2e30a11cb409 None None] MFA Rules not processed for user `ffe72dd809a84bd5aae15a85f081324e`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:04:20.495465 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-240c16a7-c1dd-4cac-9d9f-2e30a11cb409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:20.516139 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b7d37-6f42-4cf2-b4c2-a3e37ad519ca admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:20.520348 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9095a94f-23b8-4a8f-9e02-59b272ea6115 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:20.521082 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9095a94f-23b8-4a8f-9e02-59b272ea6115 service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:20.521368 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9095a94f-23b8-4a8f-9e02-59b272ea6115 service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:20.521647 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9095a94f-23b8-4a8f-9e02-59b272ea6115 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:20.531123 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9095a94f-23b8-4a8f-9e02-59b272ea6115 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:20.532596 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9095a94f-23b8-4a8f-9e02-59b272ea6115 service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:20.542957 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1334/2697] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:04:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:04:20.555868 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1364/2698] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:04:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 02:04:20.848205 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9095a94f-23b8-4a8f-9e02-59b272ea6115 service neutron] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:20.849782 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-9095a94f-23b8-4a8f-9e02-59b272ea6115 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:20.864161 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9095a94f-23b8-4a8f-9e02-59b272ea6115 service neutron] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:20.870787 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7bb32c-3c8b-4c29-92e8-907e35eac6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:20.871049 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7bb32c-3c8b-4c29-92e8-907e35eac6ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:20.871376 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7bb32c-3c8b-4c29-92e8-907e35eac6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:20.871652 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7bb32c-3c8b-4c29-92e8-907e35eac6ba admin admin] PATH_INFO: `/v3/projects/7a34f672225d4cf7af784445aec2d6a9` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:20.872893 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7bb32c-3c8b-4c29-92e8-907e35eac6ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a34f672225d4cf7af784445aec2d6a9)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:20.874847 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7bb32c-3c8b-4c29-92e8-907e35eac6ba admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:20.892455 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:04:20.932142 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-1c7bb32c-3c8b-4c29-92e8-907e35eac6ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a34f672225d4cf7af784445aec2d6a9', 'request_id': 'req-1c7bb32c-3c8b-4c29-92e8-907e35eac6ba'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:20.938719 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-1c7bb32c-3c8b-4c29-92e8-907e35eac6ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a34f672225d4cf7af784445aec2d6a9', 'request_id': 'req-1c7bb32c-3c8b-4c29-92e8-907e35eac6ba'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:20.946841 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-1c7bb32c-3c8b-4c29-92e8-907e35eac6ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a34f672225d4cf7af784445aec2d6a9', 'request_id': 'req-1c7bb32c-3c8b-4c29-92e8-907e35eac6ba'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:20.958099 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-1c7bb32c-3c8b-4c29-92e8-907e35eac6ba admin admin] The token cache is being invalidate because project 7a34f672225d4cf7af784445aec2d6a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 02:04:20.958387 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-1c7bb32c-3c8b-4c29-92e8-907e35eac6ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:20.959013 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-1c7bb32c-3c8b-4c29-92e8-907e35eac6ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:20.960649 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1335/2699] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 02:04:20 2022] DELETE /identity/v3/projects/7a34f672225d4cf7af784445aec2d6a9 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:04:20.965600 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e996e5c7-c1c4-4063-bb54-c74ded0408d2 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:20.965887 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e996e5c7-c1c4-4063-bb54-c74ded0408d2 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:20.966215 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e996e5c7-c1c4-4063-bb54-c74ded0408d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:20.985858 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-e996e5c7-c1c4-4063-bb54-c74ded0408d2 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:04:20.987103 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-e996e5c7-c1c4-4063-bb54-c74ded0408d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:21.052584 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1365/2700] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:04:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:04:21.060465 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c5292c-09c6-42ec-bc88-9504c9f304cf None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:21.060784 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c5292c-09c6-42ec-bc88-9504c9f304cf None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:21.061062 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c5292c-09c6-42ec-bc88-9504c9f304cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:21.080118 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-f4c5292c-09c6-42ec-bc88-9504c9f304cf None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:04:21.081488 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-f4c5292c-09c6-42ec-bc88-9504c9f304cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:21.090545 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1336/2701] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:04:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:04:21.094606 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e996e5c7-c1c4-4063-bb54-c74ded0408d2 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:21.105077 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e996e5c7-c1c4-4063-bb54-c74ded0408d2 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:21.109951 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c0d92-6b3f-4f89-a013-39cc75cbb4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:21.110632 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c0d92-6b3f-4f89-a013-39cc75cbb4f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:21.110919 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c0d92-6b3f-4f89-a013-39cc75cbb4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:21.111195 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c0d92-6b3f-4f89-a013-39cc75cbb4f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:21.111959 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316c0d92-6b3f-4f89-a013-39cc75cbb4f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 02:04:21.112260 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316c0d92-6b3f-4f89-a013-39cc75cbb4f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:21.113876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316c0d92-6b3f-4f89-a013-39cc75cbb4f8 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:21.123181 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:04:21.128171 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1366/2702] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 02:04:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 02:04:21.132660 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c5292c-09c6-42ec-bc88-9504c9f304cf None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:21.142590 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c5292c-09c6-42ec-bc88-9504c9f304cf None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:21.148334 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955bddd1-272d-457c-8082-de8473f90146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:21.149045 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955bddd1-272d-457c-8082-de8473f90146 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:21.149329 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955bddd1-272d-457c-8082-de8473f90146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:21.149835 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955bddd1-272d-457c-8082-de8473f90146 admin admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:21.150333 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955bddd1-272d-457c-8082-de8473f90146 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:21.152416 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955bddd1-272d-457c-8082-de8473f90146 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:21.183852 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1337/2703] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 02:04:21 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:04:21.185509 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c0d92-6b3f-4f89-a013-39cc75cbb4f8 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:21.196103 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c0d92-6b3f-4f89-a013-39cc75cbb4f8 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:21.276941 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0f64b6-d3ab-4513-a060-aaadaba4645b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:21.277616 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0f64b6-d3ab-4513-a060-aaadaba4645b admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:21.277899 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0f64b6-d3ab-4513-a060-aaadaba4645b admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:21.278300 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0f64b6-d3ab-4513-a060-aaadaba4645b admin admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:21.279090 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0f64b6-d3ab-4513-a060-aaadaba4645b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:21.281162 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0f64b6-d3ab-4513-a060-aaadaba4645b admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:21.309914 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1367/2704] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 02:04:21 2022] POST /identity/v3/users => generated 293 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:04:21.310461 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955bddd1-272d-457c-8082-de8473f90146 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:21.320998 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955bddd1-272d-457c-8082-de8473f90146 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:21.325876 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a28e82c-e4be-4de9-b066-c5fcd8e2c379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:21.326609 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28e82c-e4be-4de9-b066-c5fcd8e2c379 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:21.327019 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28e82c-e4be-4de9-b066-c5fcd8e2c379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:21.327400 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28e82c-e4be-4de9-b066-c5fcd8e2c379 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:21.328256 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a28e82c-e4be-4de9-b066-c5fcd8e2c379 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:21.330220 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a28e82c-e4be-4de9-b066-c5fcd8e2c379 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:21.343277 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1338/2705] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:04:21.345720 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0f64b6-d3ab-4513-a060-aaadaba4645b admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:21.356016 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0f64b6-d3ab-4513-a060-aaadaba4645b admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:21.360703 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516b14ab-2ed4-4072-b318-84d39b1ba259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:21.361363 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b14ab-2ed4-4072-b318-84d39b1ba259 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:21.361646 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b14ab-2ed4-4072-b318-84d39b1ba259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:21.361921 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b14ab-2ed4-4072-b318-84d39b1ba259 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:21.362656 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516b14ab-2ed4-4072-b318-84d39b1ba259 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:21.364297 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516b14ab-2ed4-4072-b318-84d39b1ba259 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:21.374480 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1368/2706] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:04:21.378706 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a28e82c-e4be-4de9-b066-c5fcd8e2c379 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:21.388647 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a28e82c-e4be-4de9-b066-c5fcd8e2c379 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:21.393351 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c4e21-7fa9-47d2-95a5-f038527c21c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:21.394240 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c4e21-7fa9-47d2-95a5-f038527c21c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:21.394520 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c4e21-7fa9-47d2-95a5-f038527c21c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:21.394796 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c4e21-7fa9-47d2-95a5-f038527c21c1 admin admin] PATH_INFO: `/v3/projects/4c1b4da0367e479aab61f8773c38cbbf/users/b08bcbafc7b44a4f9a6462d279529fef/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:21.411564 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047c4e21-7fa9-47d2-95a5-f038527c21c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c1b4da0367e479aab61f8773c38cbbf, user_id=b08bcbafc7b44a4f9a6462d279529fef, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:21.414288 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047c4e21-7fa9-47d2-95a5-f038527c21c1 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:21.433642 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1339/2707] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 02:04:21 2022] PUT /identity/v3/projects/4c1b4da0367e479aab61f8773c38cbbf/users/b08bcbafc7b44a4f9a6462d279529fef/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 02:04:21.444413 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac18524-ab77-474b-ae31-c8fac89c7e51 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:21.444705 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac18524-ab77-474b-ae31-c8fac89c7e51 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:21.445001 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac18524-ab77-474b-ae31-c8fac89c7e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:21.472847 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-9ac18524-ab77-474b-ae31-c8fac89c7e51 None None] MFA Rules not processed for user `b08bcbafc7b44a4f9a6462d279529fef`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:04:21.474082 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-9ac18524-ab77-474b-ae31-c8fac89c7e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:21.539341 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1369/2708] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:04:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 02:04:21.552043 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c4e21-7fa9-47d2-95a5-f038527c21c1 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:21.554303 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-047c4e21-7fa9-47d2-95a5-f038527c21c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:21.612343 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c4e21-7fa9-47d2-95a5-f038527c21c1 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:21.616151 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190aa7e2-d726-48a6-a088-c1dcd05f47e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:21.616868 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190aa7e2-d726-48a6-a088-c1dcd05f47e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:21.617246 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190aa7e2-d726-48a6-a088-c1dcd05f47e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:21.617649 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190aa7e2-d726-48a6-a088-c1dcd05f47e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:21.626778 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190aa7e2-d726-48a6-a088-c1dcd05f47e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:21.628251 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190aa7e2-d726-48a6-a088-c1dcd05f47e6 service cinder] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:21.637265 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1340/2709] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:04:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 02:04:29.046279 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac18524-ab77-474b-ae31-c8fac89c7e51 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:29.111609 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac18524-ab77-474b-ae31-c8fac89c7e51 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:29.116381 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeaf4e4-dc7b-4b02-86d0-7cf2b7e35ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:29.117277 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeaf4e4-dc7b-4b02-86d0-7cf2b7e35ae0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:29.117554 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeaf4e4-dc7b-4b02-86d0-7cf2b7e35ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:29.117825 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeaf4e4-dc7b-4b02-86d0-7cf2b7e35ae0 admin admin] PATH_INFO: `/v3/users/b08bcbafc7b44a4f9a6462d279529fef` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:29.119042 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeaf4e4-dc7b-4b02-86d0-7cf2b7e35ae0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b08bcbafc7b44a4f9a6462d279529fef)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:29.121016 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeaf4e4-dc7b-4b02-86d0-7cf2b7e35ae0 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:29.132624 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:04:29.174111 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-ebeaf4e4-dc7b-4b02-86d0-7cf2b7e35ae0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b08bcbafc7b44a4f9a6462d279529fef'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:29.188270 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-ebeaf4e4-dc7b-4b02-86d0-7cf2b7e35ae0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b08bcbafc7b44a4f9a6462d279529fef'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:29.196770 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-ebeaf4e4-dc7b-4b02-86d0-7cf2b7e35ae0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b08bcbafc7b44a4f9a6462d279529fef'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:29.204737 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-ebeaf4e4-dc7b-4b02-86d0-7cf2b7e35ae0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b08bcbafc7b44a4f9a6462d279529fef'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:29.205399 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-ebeaf4e4-dc7b-4b02-86d0-7cf2b7e35ae0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b08bcbafc7b44a4f9a6462d279529fef'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:29.211182 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1370/2710] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 02:04:29 2022] DELETE /identity/v3/users/b08bcbafc7b44a4f9a6462d279529fef => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:04:29.215718 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e5526-304c-47e5-b729-6d6402ea36b9 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:29.216289 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e5526-304c-47e5-b729-6d6402ea36b9 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:29.216390 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e5526-304c-47e5-b729-6d6402ea36b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:29.233276 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-ae2e5526-304c-47e5-b729-6d6402ea36b9 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:04:29.234527 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-ae2e5526-304c-47e5-b729-6d6402ea36b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:29.272625 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:04:29.300599 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1341/2711] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:04:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:04:29.310104 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeaf4e4-dc7b-4b02-86d0-7cf2b7e35ae0 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:29.311519 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-ebeaf4e4-dc7b-4b02-86d0-7cf2b7e35ae0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:29.370026 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeaf4e4-dc7b-4b02-86d0-7cf2b7e35ae0 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:29.373798 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792d1e9c-f072-412f-b555-034b6e9117df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:29.374449 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d1e9c-f072-412f-b555-034b6e9117df service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:29.374734 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d1e9c-f072-412f-b555-034b6e9117df service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:29.375011 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d1e9c-f072-412f-b555-034b6e9117df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:29.384698 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792d1e9c-f072-412f-b555-034b6e9117df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:29.386149 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792d1e9c-f072-412f-b555-034b6e9117df service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:29.396423 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1371/2712] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:04:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:04:29.708233 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e5526-304c-47e5-b729-6d6402ea36b9 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:29.709775 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-ae2e5526-304c-47e5-b729-6d6402ea36b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:29.722349 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e5526-304c-47e5-b729-6d6402ea36b9 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:29.727348 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c8b2e3-c6d8-480d-8b02-068288906b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:29.728249 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8b2e3-c6d8-480d-8b02-068288906b8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:29.728526 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8b2e3-c6d8-480d-8b02-068288906b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:29.728776 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c8b2e3-c6d8-480d-8b02-068288906b8b admin admin] PATH_INFO: `/v3/projects/4c1b4da0367e479aab61f8773c38cbbf` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:29.730000 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c8b2e3-c6d8-480d-8b02-068288906b8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c1b4da0367e479aab61f8773c38cbbf)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:29.732033 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c8b2e3-c6d8-480d-8b02-068288906b8b admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:29.781584 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-71c8b2e3-c6d8-480d-8b02-068288906b8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c1b4da0367e479aab61f8773c38cbbf', 'request_id': 'req-71c8b2e3-c6d8-480d-8b02-068288906b8b'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:29.787548 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-71c8b2e3-c6d8-480d-8b02-068288906b8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c1b4da0367e479aab61f8773c38cbbf', 'request_id': 'req-71c8b2e3-c6d8-480d-8b02-068288906b8b'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:29.795870 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-71c8b2e3-c6d8-480d-8b02-068288906b8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c1b4da0367e479aab61f8773c38cbbf', 'request_id': 'req-71c8b2e3-c6d8-480d-8b02-068288906b8b'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:29.806359 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-71c8b2e3-c6d8-480d-8b02-068288906b8b admin admin] The token cache is being invalidate because project 4c1b4da0367e479aab61f8773c38cbbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 02:04:29.806642 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-71c8b2e3-c6d8-480d-8b02-068288906b8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:29.807247 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-71c8b2e3-c6d8-480d-8b02-068288906b8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:29.808810 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1342/2713] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 02:04:29 2022] DELETE /identity/v3/projects/4c1b4da0367e479aab61f8773c38cbbf => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:04:29.813811 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a5670d-2461-4644-b08b-7f1e5a95e2fc None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:29.814142 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a5670d-2461-4644-b08b-7f1e5a95e2fc None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:29.814428 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a5670d-2461-4644-b08b-7f1e5a95e2fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:29.833040 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-d1a5670d-2461-4644-b08b-7f1e5a95e2fc None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:04:29.834246 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-d1a5670d-2461-4644-b08b-7f1e5a95e2fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:29.894692 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1372/2714] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:04:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 02:04:29.903615 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3292ed7-c7ef-4bfa-b6ea-3c5270e4fe8f None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:29.903907 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3292ed7-c7ef-4bfa-b6ea-3c5270e4fe8f None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:29.904183 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3292ed7-c7ef-4bfa-b6ea-3c5270e4fe8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:29.925053 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-c3292ed7-c7ef-4bfa-b6ea-3c5270e4fe8f None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:04:29.926286 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-c3292ed7-c7ef-4bfa-b6ea-3c5270e4fe8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:29.935508 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1343/2715] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:04:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:04:29.942037 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a5670d-2461-4644-b08b-7f1e5a95e2fc None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:29.953205 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a5670d-2461-4644-b08b-7f1e5a95e2fc None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:29.964070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d137f6-5760-46b7-8ae0-80fba8b59bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:29.964070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d137f6-5760-46b7-8ae0-80fba8b59bc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:29.964070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d137f6-5760-46b7-8ae0-80fba8b59bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:29.964070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d137f6-5760-46b7-8ae0-80fba8b59bc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:29.965657 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d137f6-5760-46b7-8ae0-80fba8b59bc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 02:04:29.965657 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d137f6-5760-46b7-8ae0-80fba8b59bc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:29.968321 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d137f6-5760-46b7-8ae0-80fba8b59bc0 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:29.980943 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1373/2716] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 02:04:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 02:04:29.985687 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3292ed7-c7ef-4bfa-b6ea-3c5270e4fe8f None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:29.995846 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3292ed7-c7ef-4bfa-b6ea-3c5270e4fe8f None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:30.001584 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b4b684-d0ff-4ebd-ad12-09e3e3c45105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:30.002236 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b4b684-d0ff-4ebd-ad12-09e3e3c45105 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:30.002515 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b4b684-d0ff-4ebd-ad12-09e3e3c45105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:30.002780 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b4b684-d0ff-4ebd-ad12-09e3e3c45105 admin admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:30.003500 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b4b684-d0ff-4ebd-ad12-09e3e3c45105 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:30.005421 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b4b684-d0ff-4ebd-ad12-09e3e3c45105 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:30.034639 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1344/2717] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 02:04:29 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:04:30.038893 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d137f6-5760-46b7-8ae0-80fba8b59bc0 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:30.049796 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d137f6-5760-46b7-8ae0-80fba8b59bc0 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:30.062698 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:04:30.110741 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee5371-7136-4a64-a2c0-e6673828f868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:30.111432 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee5371-7136-4a64-a2c0-e6673828f868 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:30.111765 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee5371-7136-4a64-a2c0-e6673828f868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:30.112044 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee5371-7136-4a64-a2c0-e6673828f868 admin admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:30.112817 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee5371-7136-4a64-a2c0-e6673828f868 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:30.114883 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee5371-7136-4a64-a2c0-e6673828f868 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:30.138185 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1374/2718] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 02:04:30 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:04:30.142709 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b4b684-d0ff-4ebd-ad12-09e3e3c45105 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:30.155113 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b4b684-d0ff-4ebd-ad12-09e3e3c45105 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:30.159991 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b037f0-8c71-4258-82bd-0398c69f62af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:30.160651 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b037f0-8c71-4258-82bd-0398c69f62af admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:30.160926 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b037f0-8c71-4258-82bd-0398c69f62af admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:30.161321 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b037f0-8c71-4258-82bd-0398c69f62af admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:30.162059 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b037f0-8c71-4258-82bd-0398c69f62af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:30.163688 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b037f0-8c71-4258-82bd-0398c69f62af admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:30.175931 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1345/2719] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:04:30.180182 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee5371-7136-4a64-a2c0-e6673828f868 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:30.190250 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee5371-7136-4a64-a2c0-e6673828f868 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:30.194891 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5c3572-9e51-4075-9925-a75852d6cf35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:30.195565 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5c3572-9e51-4075-9925-a75852d6cf35 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:30.195846 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5c3572-9e51-4075-9925-a75852d6cf35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:30.196120 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5c3572-9e51-4075-9925-a75852d6cf35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:30.196843 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5c3572-9e51-4075-9925-a75852d6cf35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:30.198446 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5c3572-9e51-4075-9925-a75852d6cf35 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:30.211262 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1375/2720] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:04:30.215394 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b037f0-8c71-4258-82bd-0398c69f62af admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:30.225496 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b037f0-8c71-4258-82bd-0398c69f62af admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:30.231566 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b087d195-f87d-40f1-bf98-d81dda1e7edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:30.232451 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b087d195-f87d-40f1-bf98-d81dda1e7edc admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:30.232721 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b087d195-f87d-40f1-bf98-d81dda1e7edc admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:30.232992 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b087d195-f87d-40f1-bf98-d81dda1e7edc admin admin] PATH_INFO: `/v3/projects/fa384bb14c7b47fca32da88f85c8c025/users/9f69cc9e2af04436872440634482430b/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:30.252376 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b087d195-f87d-40f1-bf98-d81dda1e7edc admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa384bb14c7b47fca32da88f85c8c025, user_id=9f69cc9e2af04436872440634482430b, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:30.256521 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b087d195-f87d-40f1-bf98-d81dda1e7edc admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:30.274488 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1346/2721] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 02:04:30 2022] PUT /identity/v3/projects/fa384bb14c7b47fca32da88f85c8c025/users/9f69cc9e2af04436872440634482430b/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:04:30.285247 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992fb0f-8df3-43ad-b8be-efd691df83fa None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:30.285452 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992fb0f-8df3-43ad-b8be-efd691df83fa None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:30.285750 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992fb0f-8df3-43ad-b8be-efd691df83fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:30.319017 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-7992fb0f-8df3-43ad-b8be-efd691df83fa None None] MFA Rules not processed for user `9f69cc9e2af04436872440634482430b`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:04:30.320858 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-7992fb0f-8df3-43ad-b8be-efd691df83fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:30.398234 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1376/2722] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:04:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:04:30.403335 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b087d195-f87d-40f1-bf98-d81dda1e7edc admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:30.492150 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b087d195-f87d-40f1-bf98-d81dda1e7edc admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:30.498610 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24bffbd-0202-4c59-927d-b610003ccfdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:30.498736 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24bffbd-0202-4c59-927d-b610003ccfdd admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:30.498999 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24bffbd-0202-4c59-927d-b610003ccfdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:30.499311 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24bffbd-0202-4c59-927d-b610003ccfdd admin admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:30.500010 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24bffbd-0202-4c59-927d-b610003ccfdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:30.502040 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24bffbd-0202-4c59-927d-b610003ccfdd admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:30.505962 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:04:30.533858 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1347/2723] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 02:04:30 2022] POST /identity/v3/projects => generated 375 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:04:30.535059 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7992fb0f-8df3-43ad-b8be-efd691df83fa None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:30.546359 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7992fb0f-8df3-43ad-b8be-efd691df83fa None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:30.605116 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d21ab-9ef6-40ba-a88a-956e951c6b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:30.605906 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d21ab-9ef6-40ba-a88a-956e951c6b37 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:30.606252 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d21ab-9ef6-40ba-a88a-956e951c6b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:30.606653 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d21ab-9ef6-40ba-a88a-956e951c6b37 admin admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:30.607445 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2d21ab-9ef6-40ba-a88a-956e951c6b37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:30.609603 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2d21ab-9ef6-40ba-a88a-956e951c6b37 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:30.638415 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1377/2724] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 02:04:30 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:04:30.642723 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24bffbd-0202-4c59-927d-b610003ccfdd admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:30.654631 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24bffbd-0202-4c59-927d-b610003ccfdd admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:30.659492 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baadb199-1af2-4684-a6d0-2b28e3084e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:30.660009 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baadb199-1af2-4684-a6d0-2b28e3084e20 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:30.660295 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baadb199-1af2-4684-a6d0-2b28e3084e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:30.660628 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baadb199-1af2-4684-a6d0-2b28e3084e20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:30.661407 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baadb199-1af2-4684-a6d0-2b28e3084e20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:30.662970 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baadb199-1af2-4684-a6d0-2b28e3084e20 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:30.674458 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1348/2725] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 02:04:30.678674 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d21ab-9ef6-40ba-a88a-956e951c6b37 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:30.689204 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d21ab-9ef6-40ba-a88a-956e951c6b37 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:30.693890 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91060e42-31d7-4b7b-8152-917b87159c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:30.694546 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91060e42-31d7-4b7b-8152-917b87159c11 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:30.694820 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91060e42-31d7-4b7b-8152-917b87159c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:30.695115 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91060e42-31d7-4b7b-8152-917b87159c11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:30.695934 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91060e42-31d7-4b7b-8152-917b87159c11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:30.697494 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91060e42-31d7-4b7b-8152-917b87159c11 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:30.708069 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1378/2726] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:04:30.712174 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baadb199-1af2-4684-a6d0-2b28e3084e20 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:30.722318 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baadb199-1af2-4684-a6d0-2b28e3084e20 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:30.726843 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a78cff-137d-47b6-9bcb-c9687b2c4729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:30.727756 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a78cff-137d-47b6-9bcb-c9687b2c4729 admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:30.728039 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a78cff-137d-47b6-9bcb-c9687b2c4729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:30.728314 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a78cff-137d-47b6-9bcb-c9687b2c4729 admin admin] PATH_INFO: `/v3/projects/4ee3bd8e31c04658bedfd66ea80d11c1/users/4c7a9ab86067497fa5946bc294247aed/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:30.743254 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a78cff-137d-47b6-9bcb-c9687b2c4729 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ee3bd8e31c04658bedfd66ea80d11c1, user_id=4c7a9ab86067497fa5946bc294247aed, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:30.745950 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a78cff-137d-47b6-9bcb-c9687b2c4729 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:30.759571 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1349/2727] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 02:04:30 2022] PUT /identity/v3/projects/4ee3bd8e31c04658bedfd66ea80d11c1/users/4c7a9ab86067497fa5946bc294247aed/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:04:30.771040 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40222644-a8ec-4df3-93f6-dfa0fe33ff7c None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:30.771382 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40222644-a8ec-4df3-93f6-dfa0fe33ff7c None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:30.771715 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40222644-a8ec-4df3-93f6-dfa0fe33ff7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:30.779343 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:04:30.799645 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-40222644-a8ec-4df3-93f6-dfa0fe33ff7c None None] MFA Rules not processed for user `4c7a9ab86067497fa5946bc294247aed`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:04:30.800790 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-40222644-a8ec-4df3-93f6-dfa0fe33ff7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:30.873550 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1379/2728] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:04:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:04:30.878167 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a78cff-137d-47b6-9bcb-c9687b2c4729 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:30.940992 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a78cff-137d-47b6-9bcb-c9687b2c4729 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:30.946009 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c5b81-887f-4513-a4ed-1488892187b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:30.946667 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c5b81-887f-4513-a4ed-1488892187b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:30.946951 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c5b81-887f-4513-a4ed-1488892187b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:30.947238 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c5b81-887f-4513-a4ed-1488892187b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:30.947944 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8c5b81-887f-4513-a4ed-1488892187b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:30.949837 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8c5b81-887f-4513-a4ed-1488892187b8 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:30.977151 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1350/2729] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 02:04:30 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:04:30.982114 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40222644-a8ec-4df3-93f6-dfa0fe33ff7c None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:30.997393 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40222644-a8ec-4df3-93f6-dfa0fe33ff7c None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:31.055704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328cba73-6c17-421d-8252-c28215a075d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:31.056348 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328cba73-6c17-421d-8252-c28215a075d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:31.056702 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328cba73-6c17-421d-8252-c28215a075d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:31.057226 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328cba73-6c17-421d-8252-c28215a075d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:31.057953 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328cba73-6c17-421d-8252-c28215a075d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:31.060113 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328cba73-6c17-421d-8252-c28215a075d1 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:31.080387 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1380/2730] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 02:04:30 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:04:31.084901 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c5b81-887f-4513-a4ed-1488892187b8 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:31.095955 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c5b81-887f-4513-a4ed-1488892187b8 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:31.100748 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8b74a5-b591-48b4-8c39-1e5d936541e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:31.101449 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b74a5-b591-48b4-8c39-1e5d936541e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:31.101726 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b74a5-b591-48b4-8c39-1e5d936541e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:31.101999 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b74a5-b591-48b4-8c39-1e5d936541e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:31.102753 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8b74a5-b591-48b4-8c39-1e5d936541e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:31.104423 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8b74a5-b591-48b4-8c39-1e5d936541e7 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:31.115110 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1351/2731] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:04:31.119495 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328cba73-6c17-421d-8252-c28215a075d1 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:31.129410 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328cba73-6c17-421d-8252-c28215a075d1 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:31.134038 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17296eef-cce1-44c5-8cfb-59a2b1e7b58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:31.134829 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17296eef-cce1-44c5-8cfb-59a2b1e7b58d admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:31.135119 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17296eef-cce1-44c5-8cfb-59a2b1e7b58d admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:31.135406 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17296eef-cce1-44c5-8cfb-59a2b1e7b58d admin admin] PATH_INFO: `/v3/projects/bc0696e2a38042fc99472ec8f5fee1b0/users/5eddc294ea354a3e85a73467f3368f66/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:31.150373 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17296eef-cce1-44c5-8cfb-59a2b1e7b58d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc0696e2a38042fc99472ec8f5fee1b0, user_id=5eddc294ea354a3e85a73467f3368f66, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:31.153144 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17296eef-cce1-44c5-8cfb-59a2b1e7b58d admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:31.168063 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1381/2732] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 02:04:31 2022] PUT /identity/v3/projects/bc0696e2a38042fc99472ec8f5fee1b0/users/5eddc294ea354a3e85a73467f3368f66/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 02:04:31.172062 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8b74a5-b591-48b4-8c39-1e5d936541e7 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:31.182265 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8b74a5-b591-48b4-8c39-1e5d936541e7 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:31.246074 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c6b75a-8924-426a-897f-d3244a895194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:31.246805 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6b75a-8924-426a-897f-d3244a895194 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:31.247021 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6b75a-8924-426a-897f-d3244a895194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:31.247393 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6b75a-8924-426a-897f-d3244a895194 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:31.248355 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c6b75a-8924-426a-897f-d3244a895194 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:31.250071 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c6b75a-8924-426a-897f-d3244a895194 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:31.264370 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1352/2733] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 02:04:31.267836 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17296eef-cce1-44c5-8cfb-59a2b1e7b58d admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:31.280989 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17296eef-cce1-44c5-8cfb-59a2b1e7b58d admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:31.285885 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a368c8bc-f74c-4561-abfe-3b5bd2640e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:31.286808 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368c8bc-f74c-4561-abfe-3b5bd2640e4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:31.287086 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368c8bc-f74c-4561-abfe-3b5bd2640e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:31.287394 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368c8bc-f74c-4561-abfe-3b5bd2640e4d admin admin] PATH_INFO: `/v3/projects/bc0696e2a38042fc99472ec8f5fee1b0/users/5eddc294ea354a3e85a73467f3368f66/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:31.289496 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a368c8bc-f74c-4561-abfe-3b5bd2640e4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc0696e2a38042fc99472ec8f5fee1b0, user_id=5eddc294ea354a3e85a73467f3368f66, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:31.292593 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a368c8bc-f74c-4561-abfe-3b5bd2640e4d admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:31.306417 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1382/2734] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 02:04:31 2022] PUT /identity/v3/projects/bc0696e2a38042fc99472ec8f5fee1b0/users/5eddc294ea354a3e85a73467f3368f66/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:04:31.317275 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7c1ea-3c62-4db9-8a07-b0cb97d8757c None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:31.317557 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7c1ea-3c62-4db9-8a07-b0cb97d8757c None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:31.317825 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7c1ea-3c62-4db9-8a07-b0cb97d8757c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:31.345223 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-7ec7c1ea-3c62-4db9-8a07-b0cb97d8757c None None] MFA Rules not processed for user `5eddc294ea354a3e85a73467f3368f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:04:31.346430 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-7ec7c1ea-3c62-4db9-8a07-b0cb97d8757c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:31.354773 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:04:31.419732 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1353/2735] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:04:31 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:04:31.430983 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a368c8bc-f74c-4561-abfe-3b5bd2640e4d admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:31.433247 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-a368c8bc-f74c-4561-abfe-3b5bd2640e4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:31.491278 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:04:31.498608 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a368c8bc-f74c-4561-abfe-3b5bd2640e4d admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:31.502881 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d072dd7-1927-4b90-82fa-87b5725a9bcb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:31.503682 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d072dd7-1927-4b90-82fa-87b5725a9bcb service cinder] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:31.503985 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d072dd7-1927-4b90-82fa-87b5725a9bcb service cinder] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:31.504259 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d072dd7-1927-4b90-82fa-87b5725a9bcb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:31.565011 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d072dd7-1927-4b90-82fa-87b5725a9bcb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:31.566386 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d072dd7-1927-4b90-82fa-87b5725a9bcb service cinder] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:31.575247 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1383/2736] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:04:31 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:04:33.091980 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec7c1ea-3c62-4db9-8a07-b0cb97d8757c None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:33.093530 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-7ec7c1ea-3c62-4db9-8a07-b0cb97d8757c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:33.106083 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec7c1ea-3c62-4db9-8a07-b0cb97d8757c None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:33.109775 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c81b73-a3f5-4211-a600-3cf93176807f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:33.110422 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c81b73-a3f5-4211-a600-3cf93176807f service cinder] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:33.110705 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c81b73-a3f5-4211-a600-3cf93176807f service cinder] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:33.110979 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c81b73-a3f5-4211-a600-3cf93176807f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:33.169249 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c81b73-a3f5-4211-a600-3cf93176807f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:33.170806 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c81b73-a3f5-4211-a600-3cf93176807f service cinder] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:33.180353 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1354/2737] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:04:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:04:33.723577 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d072dd7-1927-4b90-82fa-87b5725a9bcb service cinder] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:33.732927 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d072dd7-1927-4b90-82fa-87b5725a9bcb service cinder] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:33.737138 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0febdec-a203-4ade-90c1-26081f83ede7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:33.737784 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0febdec-a203-4ade-90c1-26081f83ede7 service cinder] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:33.738063 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0febdec-a203-4ade-90c1-26081f83ede7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:33.738333 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0febdec-a203-4ade-90c1-26081f83ede7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:33.748660 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0febdec-a203-4ade-90c1-26081f83ede7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:33.750152 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0febdec-a203-4ade-90c1-26081f83ede7 service cinder] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:33.761226 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1384/2738] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:04:33 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:04:37.660549 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c81b73-a3f5-4211-a600-3cf93176807f service cinder] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:37.736541 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c81b73-a3f5-4211-a600-3cf93176807f service cinder] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:37.742255 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59977841-065a-4440-9cb3-7200e178d920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:37.743124 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59977841-065a-4440-9cb3-7200e178d920 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:37.743486 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59977841-065a-4440-9cb3-7200e178d920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:37.743767 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59977841-065a-4440-9cb3-7200e178d920 admin admin] PATH_INFO: `/v3/users/9f69cc9e2af04436872440634482430b` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:37.745072 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59977841-065a-4440-9cb3-7200e178d920 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f69cc9e2af04436872440634482430b)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:37.747191 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59977841-065a-4440-9cb3-7200e178d920 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:37.798247 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:04:37.800016 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-59977841-065a-4440-9cb3-7200e178d920 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f69cc9e2af04436872440634482430b'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:37.813096 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-59977841-065a-4440-9cb3-7200e178d920 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f69cc9e2af04436872440634482430b'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:37.821892 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-59977841-065a-4440-9cb3-7200e178d920 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f69cc9e2af04436872440634482430b'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:37.829332 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-59977841-065a-4440-9cb3-7200e178d920 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f69cc9e2af04436872440634482430b'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:37.830043 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-59977841-065a-4440-9cb3-7200e178d920 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f69cc9e2af04436872440634482430b'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:37.836393 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1355/2739] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 02:04:37 2022] DELETE /identity/v3/users/9f69cc9e2af04436872440634482430b => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:04:37.841047 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53344ff3-71b9-4496-b545-9afc08f035a7 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:37.841527 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53344ff3-71b9-4496-b545-9afc08f035a7 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:37.841631 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53344ff3-71b9-4496-b545-9afc08f035a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:37.860145 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-53344ff3-71b9-4496-b545-9afc08f035a7 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:04:37.861160 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-53344ff3-71b9-4496-b545-9afc08f035a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:37.927430 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1385/2740] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:04:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:04:37.937448 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59977841-065a-4440-9cb3-7200e178d920 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:37.938956 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-59977841-065a-4440-9cb3-7200e178d920 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:38.005052 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59977841-065a-4440-9cb3-7200e178d920 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:38.012139 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aef389a-72c5-47ec-935f-81eced3f0f98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:38.013010 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef389a-72c5-47ec-935f-81eced3f0f98 service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:38.013288 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef389a-72c5-47ec-935f-81eced3f0f98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:38.013560 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef389a-72c5-47ec-935f-81eced3f0f98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:38.025249 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aef389a-72c5-47ec-935f-81eced3f0f98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:38.026691 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aef389a-72c5-47ec-935f-81eced3f0f98 service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:38.036677 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1356/2741] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:04:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:04:38.312890 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53344ff3-71b9-4496-b545-9afc08f035a7 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:38.314548 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-53344ff3-71b9-4496-b545-9afc08f035a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:38.327864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53344ff3-71b9-4496-b545-9afc08f035a7 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:38.333001 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39e3c85-0836-450d-8b1f-b33dfe974cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:38.333954 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e3c85-0836-450d-8b1f-b33dfe974cc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:38.334233 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e3c85-0836-450d-8b1f-b33dfe974cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:38.334501 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e3c85-0836-450d-8b1f-b33dfe974cc4 admin admin] PATH_INFO: `/v3/projects/fa384bb14c7b47fca32da88f85c8c025` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:38.335850 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39e3c85-0836-450d-8b1f-b33dfe974cc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa384bb14c7b47fca32da88f85c8c025)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:38.337800 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39e3c85-0836-450d-8b1f-b33dfe974cc4 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:38.385413 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-e39e3c85-0836-450d-8b1f-b33dfe974cc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa384bb14c7b47fca32da88f85c8c025', 'request_id': 'req-e39e3c85-0836-450d-8b1f-b33dfe974cc4'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:38.391359 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-e39e3c85-0836-450d-8b1f-b33dfe974cc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa384bb14c7b47fca32da88f85c8c025', 'request_id': 'req-e39e3c85-0836-450d-8b1f-b33dfe974cc4'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:38.399895 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-e39e3c85-0836-450d-8b1f-b33dfe974cc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa384bb14c7b47fca32da88f85c8c025', 'request_id': 'req-e39e3c85-0836-450d-8b1f-b33dfe974cc4'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:38.410619 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-e39e3c85-0836-450d-8b1f-b33dfe974cc4 admin admin] The token cache is being invalidate because project fa384bb14c7b47fca32da88f85c8c025 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 02:04:38.410902 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-e39e3c85-0836-450d-8b1f-b33dfe974cc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:38.411531 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-e39e3c85-0836-450d-8b1f-b33dfe974cc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:38.413069 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1386/2742] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 02:04:38 2022] DELETE /identity/v3/projects/fa384bb14c7b47fca32da88f85c8c025 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:04:38.417478 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aef389a-72c5-47ec-935f-81eced3f0f98 service neutron] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:38.427662 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aef389a-72c5-47ec-935f-81eced3f0f98 service neutron] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:38.492241 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b35bd7-9a91-4d2b-983b-4d6ee2149692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:38.493359 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b35bd7-9a91-4d2b-983b-4d6ee2149692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:38.493716 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b35bd7-9a91-4d2b-983b-4d6ee2149692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:38.494054 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b35bd7-9a91-4d2b-983b-4d6ee2149692 admin admin] PATH_INFO: `/v3/users/4c7a9ab86067497fa5946bc294247aed` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:38.496065 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b35bd7-9a91-4d2b-983b-4d6ee2149692 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c7a9ab86067497fa5946bc294247aed)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:38.498546 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b35bd7-9a91-4d2b-983b-4d6ee2149692 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:38.549126 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-c9b35bd7-9a91-4d2b-983b-4d6ee2149692 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c7a9ab86067497fa5946bc294247aed'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:38.562958 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-c9b35bd7-9a91-4d2b-983b-4d6ee2149692 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c7a9ab86067497fa5946bc294247aed'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:38.565576 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:04:38.571933 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-c9b35bd7-9a91-4d2b-983b-4d6ee2149692 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c7a9ab86067497fa5946bc294247aed'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:38.580460 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-c9b35bd7-9a91-4d2b-983b-4d6ee2149692 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c7a9ab86067497fa5946bc294247aed'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:38.581296 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-c9b35bd7-9a91-4d2b-983b-4d6ee2149692 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c7a9ab86067497fa5946bc294247aed'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:38.588315 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1357/2743] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 02:04:38 2022] DELETE /identity/v3/users/4c7a9ab86067497fa5946bc294247aed => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:04:38.876107 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39e3c85-0836-450d-8b1f-b33dfe974cc4 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:38.877611 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-e39e3c85-0836-450d-8b1f-b33dfe974cc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:38.892380 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39e3c85-0836-450d-8b1f-b33dfe974cc4 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:38.902412 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:04:38.955633 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c3cd4a-4f0c-481f-8bd5-e6157c76f557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:38.956569 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3cd4a-4f0c-481f-8bd5-e6157c76f557 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:38.956965 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3cd4a-4f0c-481f-8bd5-e6157c76f557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:38.957225 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3cd4a-4f0c-481f-8bd5-e6157c76f557 admin admin] PATH_INFO: `/v3/projects/4ee3bd8e31c04658bedfd66ea80d11c1` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:38.958456 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c3cd4a-4f0c-481f-8bd5-e6157c76f557 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ee3bd8e31c04658bedfd66ea80d11c1)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:38.960574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c3cd4a-4f0c-481f-8bd5-e6157c76f557 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:39.011463 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-79c3cd4a-4f0c-481f-8bd5-e6157c76f557 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ee3bd8e31c04658bedfd66ea80d11c1', 'request_id': 'req-79c3cd4a-4f0c-481f-8bd5-e6157c76f557'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:39.017728 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-79c3cd4a-4f0c-481f-8bd5-e6157c76f557 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ee3bd8e31c04658bedfd66ea80d11c1', 'request_id': 'req-79c3cd4a-4f0c-481f-8bd5-e6157c76f557'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:39.025964 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-79c3cd4a-4f0c-481f-8bd5-e6157c76f557 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ee3bd8e31c04658bedfd66ea80d11c1', 'request_id': 'req-79c3cd4a-4f0c-481f-8bd5-e6157c76f557'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:39.038520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-79c3cd4a-4f0c-481f-8bd5-e6157c76f557 admin admin] The token cache is being invalidate because project 4ee3bd8e31c04658bedfd66ea80d11c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 02:04:39.038811 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-79c3cd4a-4f0c-481f-8bd5-e6157c76f557 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:39.039552 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-79c3cd4a-4f0c-481f-8bd5-e6157c76f557 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:39.041553 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1387/2744] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 02:04:38 2022] DELETE /identity/v3/projects/4ee3bd8e31c04658bedfd66ea80d11c1 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:04:39.045732 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b35bd7-9a91-4d2b-983b-4d6ee2149692 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:39.047163 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-c9b35bd7-9a91-4d2b-983b-4d6ee2149692 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:39.060207 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b35bd7-9a91-4d2b-983b-4d6ee2149692 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:39.119851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f690e91-a23d-4a3e-af67-7e32c5c56212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:39.120872 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f690e91-a23d-4a3e-af67-7e32c5c56212 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:39.121198 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f690e91-a23d-4a3e-af67-7e32c5c56212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:39.121472 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f690e91-a23d-4a3e-af67-7e32c5c56212 admin admin] PATH_INFO: `/v3/users/5eddc294ea354a3e85a73467f3368f66` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:39.122783 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f690e91-a23d-4a3e-af67-7e32c5c56212 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5eddc294ea354a3e85a73467f3368f66)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:39.124869 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f690e91-a23d-4a3e-af67-7e32c5c56212 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:39.174815 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-5f690e91-a23d-4a3e-af67-7e32c5c56212 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5eddc294ea354a3e85a73467f3368f66'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:39.190847 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-5f690e91-a23d-4a3e-af67-7e32c5c56212 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5eddc294ea354a3e85a73467f3368f66'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:39.199762 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-5f690e91-a23d-4a3e-af67-7e32c5c56212 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5eddc294ea354a3e85a73467f3368f66'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:39.206575 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-5f690e91-a23d-4a3e-af67-7e32c5c56212 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5eddc294ea354a3e85a73467f3368f66'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:39.207195 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-5f690e91-a23d-4a3e-af67-7e32c5c56212 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5eddc294ea354a3e85a73467f3368f66'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:39.213124 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1358/2745] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 02:04:39 2022] DELETE /identity/v3/users/5eddc294ea354a3e85a73467f3368f66 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:04:39.494520 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c3cd4a-4f0c-481f-8bd5-e6157c76f557 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:39.496191 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-79c3cd4a-4f0c-481f-8bd5-e6157c76f557 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:39.509559 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c3cd4a-4f0c-481f-8bd5-e6157c76f557 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:39.570329 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f7f28-6ade-491c-974a-786e6c61a4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:39.571179 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f7f28-6ade-491c-974a-786e6c61a4e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:39.571497 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f7f28-6ade-491c-974a-786e6c61a4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:39.571769 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f7f28-6ade-491c-974a-786e6c61a4e9 admin admin] PATH_INFO: `/v3/projects/bc0696e2a38042fc99472ec8f5fee1b0` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:39.573067 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f7f28-6ade-491c-974a-786e6c61a4e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc0696e2a38042fc99472ec8f5fee1b0)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:39.575033 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f7f28-6ade-491c-974a-786e6c61a4e9 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:39.619919 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:04:39.624023 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-e61f7f28-6ade-491c-974a-786e6c61a4e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc0696e2a38042fc99472ec8f5fee1b0', 'request_id': 'req-e61f7f28-6ade-491c-974a-786e6c61a4e9'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:39.630385 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-e61f7f28-6ade-491c-974a-786e6c61a4e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc0696e2a38042fc99472ec8f5fee1b0', 'request_id': 'req-e61f7f28-6ade-491c-974a-786e6c61a4e9'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:39.639170 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-e61f7f28-6ade-491c-974a-786e6c61a4e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc0696e2a38042fc99472ec8f5fee1b0', 'request_id': 'req-e61f7f28-6ade-491c-974a-786e6c61a4e9'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:39.650359 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-e61f7f28-6ade-491c-974a-786e6c61a4e9 admin admin] The token cache is being invalidate because project bc0696e2a38042fc99472ec8f5fee1b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 02:04:39.650641 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-e61f7f28-6ade-491c-974a-786e6c61a4e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:39.651304 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-e61f7f28-6ade-491c-974a-786e6c61a4e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:04:39.652844 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1388/2746] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 02:04:39 2022] DELETE /identity/v3/projects/bc0696e2a38042fc99472ec8f5fee1b0 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:04:39.658272 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c4ed4-07ff-4eea-bd61-9aa08bf3bdd5 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:39.658596 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c4ed4-07ff-4eea-bd61-9aa08bf3bdd5 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:39.658881 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c4ed4-07ff-4eea-bd61-9aa08bf3bdd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:39.677968 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-855c4ed4-07ff-4eea-bd61-9aa08bf3bdd5 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:04:39.679173 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-855c4ed4-07ff-4eea-bd61-9aa08bf3bdd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:39.744251 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1359/2747] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:04:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:04:39.752524 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa15d47-07df-4d2d-8837-6161eaf98652 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:39.752837 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa15d47-07df-4d2d-8837-6161eaf98652 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:39.753111 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa15d47-07df-4d2d-8837-6161eaf98652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:39.772805 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-7fa15d47-07df-4d2d-8837-6161eaf98652 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:04:39.774019 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-7fa15d47-07df-4d2d-8837-6161eaf98652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:39.783617 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1389/2748] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:04:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:04:39.788099 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855c4ed4-07ff-4eea-bd61-9aa08bf3bdd5 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:39.794688 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:04:39.799286 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855c4ed4-07ff-4eea-bd61-9aa08bf3bdd5 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:39.806079 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33391736-218d-4509-a65f-688204cb1c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:39.806749 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33391736-218d-4509-a65f-688204cb1c04 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:39.807036 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33391736-218d-4509-a65f-688204cb1c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:39.807354 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33391736-218d-4509-a65f-688204cb1c04 admin admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:39.808100 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33391736-218d-4509-a65f-688204cb1c04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 02:04:39.808420 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33391736-218d-4509-a65f-688204cb1c04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:39.810430 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33391736-218d-4509-a65f-688204cb1c04 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:39.823901 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1360/2749] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 02:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 02:04:39.830524 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa15d47-07df-4d2d-8837-6161eaf98652 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:39.843997 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa15d47-07df-4d2d-8837-6161eaf98652 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:39.850106 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d30a98-37ca-412f-bd95-61412088fd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:39.850756 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d30a98-37ca-412f-bd95-61412088fd33 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:39.851034 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d30a98-37ca-412f-bd95-61412088fd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:39.851341 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d30a98-37ca-412f-bd95-61412088fd33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:39.852035 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d30a98-37ca-412f-bd95-61412088fd33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:39.854002 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d30a98-37ca-412f-bd95-61412088fd33 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:39.885929 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1390/2750] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 02:04:39 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:04:39.887084 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33391736-218d-4509-a65f-688204cb1c04 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:39.897666 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33391736-218d-4509-a65f-688204cb1c04 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:39.964272 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d75797-8fc2-4337-93d9-ad5d3664e8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:39.965178 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d75797-8fc2-4337-93d9-ad5d3664e8d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:39.965453 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d75797-8fc2-4337-93d9-ad5d3664e8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:39.965826 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d75797-8fc2-4337-93d9-ad5d3664e8d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:39.966708 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d75797-8fc2-4337-93d9-ad5d3664e8d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:39.968804 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d75797-8fc2-4337-93d9-ad5d3664e8d3 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:39.990601 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1361/2751] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 02:04:39 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 02:04:39.994954 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d30a98-37ca-412f-bd95-61412088fd33 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:40.005628 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d30a98-37ca-412f-bd95-61412088fd33 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:40.011145 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb2e53-bc44-4bf9-a4d5-3bf40256687d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:40.011837 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb2e53-bc44-4bf9-a4d5-3bf40256687d admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:40.012111 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb2e53-bc44-4bf9-a4d5-3bf40256687d admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:40.012382 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb2e53-bc44-4bf9-a4d5-3bf40256687d admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:40.013104 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeb2e53-bc44-4bf9-a4d5-3bf40256687d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:40.014743 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeb2e53-bc44-4bf9-a4d5-3bf40256687d admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:40.025531 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1391/2752] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:04:40.030128 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d75797-8fc2-4337-93d9-ad5d3664e8d3 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:40.045629 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d75797-8fc2-4337-93d9-ad5d3664e8d3 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:40.051370 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5a9eef-8c3c-467d-99d6-b2936e62b219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:40.051989 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a9eef-8c3c-467d-99d6-b2936e62b219 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:40.052298 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a9eef-8c3c-467d-99d6-b2936e62b219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:40.052573 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a9eef-8c3c-467d-99d6-b2936e62b219 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:40.053305 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5a9eef-8c3c-467d-99d6-b2936e62b219 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:40.054904 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5a9eef-8c3c-467d-99d6-b2936e62b219 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:40.065936 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1362/2753] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:04:40.070278 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb2e53-bc44-4bf9-a4d5-3bf40256687d admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:40.080644 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb2e53-bc44-4bf9-a4d5-3bf40256687d admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:40.085163 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a4615-e861-4c11-9d2d-18ba965fb84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:40.086027 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a4615-e861-4c11-9d2d-18ba965fb84e admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:40.086299 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a4615-e861-4c11-9d2d-18ba965fb84e admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:40.086566 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a4615-e861-4c11-9d2d-18ba965fb84e admin admin] PATH_INFO: `/v3/projects/136fd75d8a874bb3aa24ef1564f8e36b/users/2b55b66ac25848bda892d86d4d0d6f4e/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:40.101338 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07a4615-e861-4c11-9d2d-18ba965fb84e admin admin] RBAC: Authorizing `identity:create_grant(project_id=136fd75d8a874bb3aa24ef1564f8e36b, user_id=2b55b66ac25848bda892d86d4d0d6f4e, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:40.104034 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07a4615-e861-4c11-9d2d-18ba965fb84e admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:40.118658 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1392/2754] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 02:04:40 2022] PUT /identity/v3/projects/136fd75d8a874bb3aa24ef1564f8e36b/users/2b55b66ac25848bda892d86d4d0d6f4e/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:04:40.123885 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786706e5-4dae-421f-8a91-6022798e796d None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:40.124202 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786706e5-4dae-421f-8a91-6022798e796d None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:40.124483 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786706e5-4dae-421f-8a91-6022798e796d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:40.143704 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-786706e5-4dae-421f-8a91-6022798e796d None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:04:40.144911 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-786706e5-4dae-421f-8a91-6022798e796d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:40.206643 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1363/2755] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:04:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 02:04:40.216579 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a4615-e861-4c11-9d2d-18ba965fb84e admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:40.218446 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-b07a4615-e861-4c11-9d2d-18ba965fb84e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:40.276964 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a4615-e861-4c11-9d2d-18ba965fb84e admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:40.281139 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032a693e-cff6-4b20-9e13-41f47d769119 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:40.281771 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a693e-cff6-4b20-9e13-41f47d769119 service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:40.283451 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a693e-cff6-4b20-9e13-41f47d769119 service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:40.283736 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a693e-cff6-4b20-9e13-41f47d769119 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:40.293664 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032a693e-cff6-4b20-9e13-41f47d769119 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:40.295138 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032a693e-cff6-4b20-9e13-41f47d769119 service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:40.304862 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1393/2756] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:04:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 02:04:43.510578 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786706e5-4dae-421f-8a91-6022798e796d None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:43.512403 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-786706e5-4dae-421f-8a91-6022798e796d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:43.602835 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786706e5-4dae-421f-8a91-6022798e796d None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:43.606873 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318b7ce4-8ccc-443a-b326-892d7b86dedd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:43.607747 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b7ce4-8ccc-443a-b326-892d7b86dedd service cinder] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:43.608036 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b7ce4-8ccc-443a-b326-892d7b86dedd service cinder] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:43.608311 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b7ce4-8ccc-443a-b326-892d7b86dedd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:43.610626 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-318b7ce4-8ccc-443a-b326-892d7b86dedd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:43.653094 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:04:43.707369 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318b7ce4-8ccc-443a-b326-892d7b86dedd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:43.708962 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318b7ce4-8ccc-443a-b326-892d7b86dedd service cinder] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:43.721517 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1364/2757] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:04:43 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:04:44.316397 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12349b05-6e47-480c-9832-4ee764a7bc2f None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:44.316919 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12349b05-6e47-480c-9832-4ee764a7bc2f None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:44.316919 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12349b05-6e47-480c-9832-4ee764a7bc2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:44.347617 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-12349b05-6e47-480c-9832-4ee764a7bc2f None None] MFA Rules not processed for user `2b55b66ac25848bda892d86d4d0d6f4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:04:44.348852 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-12349b05-6e47-480c-9832-4ee764a7bc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:44.410122 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:04:44.414227 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1394/2758] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:04:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:04:44.419374 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318b7ce4-8ccc-443a-b326-892d7b86dedd service cinder] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:44.430215 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318b7ce4-8ccc-443a-b326-892d7b86dedd service cinder] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:44.434680 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123f1b31-d0c6-47a1-a3ca-22b9d98f312d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:44.435621 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f1b31-d0c6-47a1-a3ca-22b9d98f312d admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:44.435729 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f1b31-d0c6-47a1-a3ca-22b9d98f312d admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:44.436023 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f1b31-d0c6-47a1-a3ca-22b9d98f312d admin admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:44.436615 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123f1b31-d0c6-47a1-a3ca-22b9d98f312d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:44.438500 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123f1b31-d0c6-47a1-a3ca-22b9d98f312d admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:44.475661 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1365/2759] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 02:04:44 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:04:44.479447 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12349b05-6e47-480c-9832-4ee764a7bc2f None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:44.497027 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12349b05-6e47-480c-9832-4ee764a7bc2f None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:44.568160 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35958203-2487-4375-a41a-20e390c9eca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:44.568873 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35958203-2487-4375-a41a-20e390c9eca8 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:44.569157 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35958203-2487-4375-a41a-20e390c9eca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:44.569434 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35958203-2487-4375-a41a-20e390c9eca8 admin admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:44.570153 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35958203-2487-4375-a41a-20e390c9eca8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:44.572153 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35958203-2487-4375-a41a-20e390c9eca8 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:44.599444 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1395/2760] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 02:04:44 2022] POST /identity/v3/users => generated 293 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:04:44.601020 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123f1b31-d0c6-47a1-a3ca-22b9d98f312d admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:44.612581 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123f1b31-d0c6-47a1-a3ca-22b9d98f312d admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:44.618857 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b44d00-c67f-46e5-a470-0ba7b15cac23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:44.619745 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b44d00-c67f-46e5-a470-0ba7b15cac23 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:44.620003 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b44d00-c67f-46e5-a470-0ba7b15cac23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:44.620524 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b44d00-c67f-46e5-a470-0ba7b15cac23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:44.621309 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b44d00-c67f-46e5-a470-0ba7b15cac23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:44.623133 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b44d00-c67f-46e5-a470-0ba7b15cac23 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:44.638642 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1366/2761] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:04:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:04:44.640003 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35958203-2487-4375-a41a-20e390c9eca8 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:44.650646 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35958203-2487-4375-a41a-20e390c9eca8 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:44.655576 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98169e89-14d6-4d69-993c-c4333e11ba0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:44.656476 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98169e89-14d6-4d69-993c-c4333e11ba0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:44.656767 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98169e89-14d6-4d69-993c-c4333e11ba0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:44.657042 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98169e89-14d6-4d69-993c-c4333e11ba0a admin admin] PATH_INFO: `/v3/projects/1183d7efcd9847ef8bd9875f0d38a7f2/users/aa904af8376d4753b5611ebfe7653b29/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:44.674041 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98169e89-14d6-4d69-993c-c4333e11ba0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1183d7efcd9847ef8bd9875f0d38a7f2, user_id=aa904af8376d4753b5611ebfe7653b29, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:44.677169 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98169e89-14d6-4d69-993c-c4333e11ba0a admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:44.691018 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1396/2762] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 02:04:44 2022] PUT /identity/v3/projects/1183d7efcd9847ef8bd9875f0d38a7f2/users/aa904af8376d4753b5611ebfe7653b29/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:04:44.695338 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b44d00-c67f-46e5-a470-0ba7b15cac23 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:44.707379 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b44d00-c67f-46e5-a470-0ba7b15cac23 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:44.775860 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74938aa0-dc22-4737-bb5d-38c4cf01e5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:44.776733 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74938aa0-dc22-4737-bb5d-38c4cf01e5d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:44.777147 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74938aa0-dc22-4737-bb5d-38c4cf01e5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:44.777544 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74938aa0-dc22-4737-bb5d-38c4cf01e5d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:44.778550 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74938aa0-dc22-4737-bb5d-38c4cf01e5d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:44.780524 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74938aa0-dc22-4737-bb5d-38c4cf01e5d1 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:44.795942 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1367/2763] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:04:44 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:04:44.797091 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98169e89-14d6-4d69-993c-c4333e11ba0a admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:44.808544 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98169e89-14d6-4d69-993c-c4333e11ba0a admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:44.813876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18179f9-b009-4d44-a592-db79cce8cff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:44.815254 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18179f9-b009-4d44-a592-db79cce8cff2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:44.815254 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18179f9-b009-4d44-a592-db79cce8cff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:44.815500 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18179f9-b009-4d44-a592-db79cce8cff2 admin admin] PATH_INFO: `/v3/projects/1183d7efcd9847ef8bd9875f0d38a7f2/users/aa904af8376d4753b5611ebfe7653b29/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:44.817467 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18179f9-b009-4d44-a592-db79cce8cff2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1183d7efcd9847ef8bd9875f0d38a7f2, user_id=aa904af8376d4753b5611ebfe7653b29, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:44.820553 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18179f9-b009-4d44-a592-db79cce8cff2 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:44.833349 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1397/2764] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 02:04:44 2022] PUT /identity/v3/projects/1183d7efcd9847ef8bd9875f0d38a7f2/users/aa904af8376d4753b5611ebfe7653b29/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 02:04:49.223106 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2725cb3-dd3e-4ed4-8da9-4ffcb2fde86f None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:49.223714 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2725cb3-dd3e-4ed4-8da9-4ffcb2fde86f None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:49.223714 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2725cb3-dd3e-4ed4-8da9-4ffcb2fde86f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:49.253439 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-b2725cb3-dd3e-4ed4-8da9-4ffcb2fde86f None None] MFA Rules not processed for user `aa904af8376d4753b5611ebfe7653b29`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:04:49.254684 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-b2725cb3-dd3e-4ed4-8da9-4ffcb2fde86f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:49.338539 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1368/2765] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:04:49 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:04:49.356191 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18179f9-b009-4d44-a592-db79cce8cff2 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:49.358126 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-d18179f9-b009-4d44-a592-db79cce8cff2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:49.440149 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18179f9-b009-4d44-a592-db79cce8cff2 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:49.446695 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb1d324-29f8-40a6-b59e-b32387c8c108 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4e83df33b64a3ebf28f54e2d94fe63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:49.447368 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1d324-29f8-40a6-b59e-b32387c8c108 service nova] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:49.447648 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1d324-29f8-40a6-b59e-b32387c8c108 service nova] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:49.447910 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1d324-29f8-40a6-b59e-b32387c8c108 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:49.498223 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:04:49.509421 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb1d324-29f8-40a6-b59e-b32387c8c108 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:49.511179 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb1d324-29f8-40a6-b59e-b32387c8c108 service nova] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:49.521079 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1398/2766] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:04:49 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:04:49.952850 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb39d6c-414e-4987-9c4f-e3ee55ee80af None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:49.953256 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb39d6c-414e-4987-9c4f-e3ee55ee80af None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:49.953479 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb39d6c-414e-4987-9c4f-e3ee55ee80af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:49.995470 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-abb39d6c-414e-4987-9c4f-e3ee55ee80af None None] MFA Rules not processed for user `c14a81624a264ab280d46d063ae676ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:04:49.999764 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-abb39d6c-414e-4987-9c4f-e3ee55ee80af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:50.017079 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:04:50.078815 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1369/2767] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:04:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:04:50.684995 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b74658-c818-4aca-8531-731e36c40313 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:50.685447 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b74658-c818-4aca-8531-731e36c40313 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:50.685807 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b74658-c818-4aca-8531-731e36c40313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:50.709539 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-72b74658-c818-4aca-8531-731e36c40313 None None] MFA Rules not processed for user `2b55b66ac25848bda892d86d4d0d6f4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:04:50.713141 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-72b74658-c818-4aca-8531-731e36c40313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:50.722121 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1399/2768] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:04:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:04:51.585292 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb39d6c-414e-4987-9c4f-e3ee55ee80af None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:04:51.587532 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-abb39d6c-414e-4987-9c4f-e3ee55ee80af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:51.663311 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb39d6c-414e-4987-9c4f-e3ee55ee80af None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:04:51.669397 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334ced7d-78bf-4d97-8418-d6a965fb6379 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5bf3baee1b843fa89b65e5485f6bebb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:04:51.670087 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334ced7d-78bf-4d97-8418-d6a965fb6379 service placement] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:04:51.670368 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334ced7d-78bf-4d97-8418-d6a965fb6379 service placement] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:04:51.670681 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334ced7d-78bf-4d97-8418-d6a965fb6379 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:04:51.672548 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-334ced7d-78bf-4d97-8418-d6a965fb6379 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:04:51.685493 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334ced7d-78bf-4d97-8418-d6a965fb6379 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:04:51.687056 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334ced7d-78bf-4d97-8418-d6a965fb6379 service placement] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:04:51.698049 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1370/2769] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:04:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:05:00.702593 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=8 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:05:01.693052 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=51 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:05:06.488778 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b74658-c818-4aca-8531-731e36c40313 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:05:06.490418 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-72b74658-c818-4aca-8531-731e36c40313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:05:06.549249 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b74658-c818-4aca-8531-731e36c40313 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:05:06.552764 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef72590e-2f1f-439b-bcb4-bd83d50b900e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:05:06.553408 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef72590e-2f1f-439b-bcb4-bd83d50b900e service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:05:06.553685 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef72590e-2f1f-439b-bcb4-bd83d50b900e service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:05:06.553958 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef72590e-2f1f-439b-bcb4-bd83d50b900e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:05:06.555773 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-ef72590e-2f1f-439b-bcb4-bd83d50b900e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:05:06.565693 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef72590e-2f1f-439b-bcb4-bd83d50b900e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:05:06.567143 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef72590e-2f1f-439b-bcb4-bd83d50b900e service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:05:06.575822 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1400/2770] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:05:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:05:08.056585 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334ced7d-78bf-4d97-8418-d6a965fb6379 service placement] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:05:08.066221 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334ced7d-78bf-4d97-8418-d6a965fb6379 service placement] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:05:08.069744 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c10df75-2fc5-47ef-9260-10c4b60c14d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:05:08.070414 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c10df75-2fc5-47ef-9260-10c4b60c14d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:05:08.070692 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c10df75-2fc5-47ef-9260-10c4b60c14d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:05:08.070963 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c10df75-2fc5-47ef-9260-10c4b60c14d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:05:08.072788 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-0c10df75-2fc5-47ef-9260-10c4b60c14d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:05:08.156290 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c10df75-2fc5-47ef-9260-10c4b60c14d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:05:08.158319 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c10df75-2fc5-47ef-9260-10c4b60c14d8 service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:05:08.177692 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1371/2771] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:05:08 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:05:10.203936 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef72590e-2f1f-439b-bcb4-bd83d50b900e service neutron] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:05:10.217736 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef72590e-2f1f-439b-bcb4-bd83d50b900e service neutron] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:05:10.224548 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a14140-3022-41ba-a7f9-b4e55e2cd13b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4e83df33b64a3ebf28f54e2d94fe63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:05:10.225242 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a14140-3022-41ba-a7f9-b4e55e2cd13b service nova] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:05:10.225524 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a14140-3022-41ba-a7f9-b4e55e2cd13b service nova] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:05:10.225790 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a14140-3022-41ba-a7f9-b4e55e2cd13b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:05:10.227791 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-38a14140-3022-41ba-a7f9-b4e55e2cd13b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:05:10.240539 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a14140-3022-41ba-a7f9-b4e55e2cd13b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:05:10.242130 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a14140-3022-41ba-a7f9-b4e55e2cd13b service nova] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:05:10.253072 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1401/2772] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:05:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:05:18.167786 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=30 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:05:20.249011 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=32 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:05:23.596813 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216f5c7-14c4-4cd0-9a00-0f6639bafd32 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:05:23.597332 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216f5c7-14c4-4cd0-9a00-0f6639bafd32 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:05:23.597441 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216f5c7-14c4-4cd0-9a00-0f6639bafd32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:05:23.616247 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-6216f5c7-14c4-4cd0-9a00-0f6639bafd32 None None] MFA Rules not processed for user `ffe72dd809a84bd5aae15a85f081324e`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:05:23.617448 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-6216f5c7-14c4-4cd0-9a00-0f6639bafd32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:05:23.675325 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1372/2773] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:05:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:05:24.438389 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a14140-3022-41ba-a7f9-b4e55e2cd13b service nova] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:05:24.448391 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a14140-3022-41ba-a7f9-b4e55e2cd13b service nova] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:05:24.452187 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e06f2-8dbc-40e1-900c-45ce41bedfd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:05:24.452840 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e06f2-8dbc-40e1-900c-45ce41bedfd3 service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:05:24.453124 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e06f2-8dbc-40e1-900c-45ce41bedfd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:05:24.453398 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e06f2-8dbc-40e1-900c-45ce41bedfd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:05:24.463074 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9e06f2-8dbc-40e1-900c-45ce41bedfd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:05:24.464631 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9e06f2-8dbc-40e1-900c-45ce41bedfd3 service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:05:24.474341 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1402/2774] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:05:24 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:05:33.667518 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=25 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:05:34.470113 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=6 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:06:28.739932 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fbbf66-49f6-4a8b-967e-a8f22e42f027 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:06:28.740469 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fbbf66-49f6-4a8b-967e-a8f22e42f027 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:06:28.740469 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fbbf66-49f6-4a8b-967e-a8f22e42f027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:06:28.758458 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-85fbbf66-49f6-4a8b-967e-a8f22e42f027 None None] MFA Rules not processed for user `ffe72dd809a84bd5aae15a85f081324e`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:06:28.759716 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-85fbbf66-49f6-4a8b-967e-a8f22e42f027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:06:28.767562 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1373/2775] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:06:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:06:28.803529 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e06f2-8dbc-40e1-900c-45ce41bedfd3 service neutron] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:06:28.807044 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-ed9e06f2-8dbc-40e1-900c-45ce41bedfd3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:06:28.873373 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e06f2-8dbc-40e1-900c-45ce41bedfd3 service neutron] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:06:28.879842 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8039f5c6-6f8d-43eb-9f27-0c031bbb383e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c081b4cf34146ca8eb197b62a133433', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:06:28.880627 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8039f5c6-6f8d-43eb-9f27-0c031bbb383e service glance] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:06:28.880970 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8039f5c6-6f8d-43eb-9f27-0c031bbb383e service glance] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:06:28.881327 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8039f5c6-6f8d-43eb-9f27-0c031bbb383e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:06:28.883792 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-8039f5c6-6f8d-43eb-9f27-0c031bbb383e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:06:28.897958 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8039f5c6-6f8d-43eb-9f27-0c031bbb383e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:06:28.899474 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8039f5c6-6f8d-43eb-9f27-0c031bbb383e service glance] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:06:28.910875 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1403/2776] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:06:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:06:29.530060 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fbbf66-49f6-4a8b-967e-a8f22e42f027 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:06:29.539828 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fbbf66-49f6-4a8b-967e-a8f22e42f027 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:06:29.543745 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552af759-b6cd-4713-8e07-60c7e3887300 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:06:29.544364 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552af759-b6cd-4713-8e07-60c7e3887300 service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:06:29.544639 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552af759-b6cd-4713-8e07-60c7e3887300 service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:06:29.544908 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552af759-b6cd-4713-8e07-60c7e3887300 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:06:29.546688 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-552af759-b6cd-4713-8e07-60c7e3887300 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:06:29.560186 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552af759-b6cd-4713-8e07-60c7e3887300 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:06:29.561640 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552af759-b6cd-4713-8e07-60c7e3887300 service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:06:29.572994 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1374/2777] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:06:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:06:38.905659 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=26 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:06:39.568716 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=9 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:07:01.708899 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8039f5c6-6f8d-43eb-9f27-0c031bbb383e service glance] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:01.718489 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8039f5c6-6f8d-43eb-9f27-0c031bbb383e service glance] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:01.722110 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b2d9bb-b25a-4fa1-b087-bf2576618a4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:01.722744 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b2d9bb-b25a-4fa1-b087-bf2576618a4e service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:01.723021 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b2d9bb-b25a-4fa1-b087-bf2576618a4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:01.723334 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b2d9bb-b25a-4fa1-b087-bf2576618a4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:01.725207 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-d3b2d9bb-b25a-4fa1-b087-bf2576618a4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:07:01.787573 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b2d9bb-b25a-4fa1-b087-bf2576618a4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:01.789015 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b2d9bb-b25a-4fa1-b087-bf2576618a4e service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:01.799889 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1404/2778] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:07:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:07:04.895773 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552af759-b6cd-4713-8e07-60c7e3887300 service neutron] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:04.912530 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552af759-b6cd-4713-8e07-60c7e3887300 service neutron] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:04.918728 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83299a3-d4bd-49eb-a7a2-35aa0dbdefcb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5bf3baee1b843fa89b65e5485f6bebb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:04.919666 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83299a3-d4bd-49eb-a7a2-35aa0dbdefcb service placement] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:04.920137 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83299a3-d4bd-49eb-a7a2-35aa0dbdefcb service placement] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:04.920570 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83299a3-d4bd-49eb-a7a2-35aa0dbdefcb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:04.922539 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-a83299a3-d4bd-49eb-a7a2-35aa0dbdefcb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:07:04.943749 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83299a3-d4bd-49eb-a7a2-35aa0dbdefcb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:04.945289 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83299a3-d4bd-49eb-a7a2-35aa0dbdefcb service placement] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:04.956640 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1375/2779] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:07:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:07:08.141474 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b2d9bb-b25a-4fa1-b087-bf2576618a4e service neutron] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:08.144099 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-d3b2d9bb-b25a-4fa1-b087-bf2576618a4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:07:08.158779 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b2d9bb-b25a-4fa1-b087-bf2576618a4e service neutron] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:08.163618 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb22343-6777-47d6-a0bf-84fcab0e0a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:08.164502 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb22343-6777-47d6-a0bf-84fcab0e0a3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:08.164782 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb22343-6777-47d6-a0bf-84fcab0e0a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:08.165056 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb22343-6777-47d6-a0bf-84fcab0e0a3c admin admin] PATH_INFO: `/v3/users/c14a81624a264ab280d46d063ae676ac` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:08.166290 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb22343-6777-47d6-a0bf-84fcab0e0a3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c14a81624a264ab280d46d063ae676ac)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:08.168633 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb22343-6777-47d6-a0bf-84fcab0e0a3c admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:08.222258 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-fcb22343-6777-47d6-a0bf-84fcab0e0a3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c14a81624a264ab280d46d063ae676ac'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:08.234486 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-fcb22343-6777-47d6-a0bf-84fcab0e0a3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c14a81624a264ab280d46d063ae676ac'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:08.245537 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-fcb22343-6777-47d6-a0bf-84fcab0e0a3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c14a81624a264ab280d46d063ae676ac'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:08.253371 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-fcb22343-6777-47d6-a0bf-84fcab0e0a3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c14a81624a264ab280d46d063ae676ac'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:08.254002 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-fcb22343-6777-47d6-a0bf-84fcab0e0a3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c14a81624a264ab280d46d063ae676ac'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:08.259962 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1405/2780] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 02:07:08 2022] DELETE /identity/v3/users/c14a81624a264ab280d46d063ae676ac => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:07:08.444586 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83299a3-d4bd-49eb-a7a2-35aa0dbdefcb service placement] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:08.457515 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83299a3-d4bd-49eb-a7a2-35aa0dbdefcb service placement] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:08.520973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e26da-5e3e-436e-947d-c81a91404d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:08.521908 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e26da-5e3e-436e-947d-c81a91404d70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:08.522200 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e26da-5e3e-436e-947d-c81a91404d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:08.522503 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e26da-5e3e-436e-947d-c81a91404d70 admin admin] PATH_INFO: `/v3/projects/59bb100f7d034605bb875abd1a0775ec` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:08.523819 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348e26da-5e3e-436e-947d-c81a91404d70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59bb100f7d034605bb875abd1a0775ec)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:08.525806 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348e26da-5e3e-436e-947d-c81a91404d70 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:08.577928 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-348e26da-5e3e-436e-947d-c81a91404d70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59bb100f7d034605bb875abd1a0775ec', 'request_id': 'req-348e26da-5e3e-436e-947d-c81a91404d70'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:08.584526 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-348e26da-5e3e-436e-947d-c81a91404d70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59bb100f7d034605bb875abd1a0775ec', 'request_id': 'req-348e26da-5e3e-436e-947d-c81a91404d70'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:08.593196 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-348e26da-5e3e-436e-947d-c81a91404d70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59bb100f7d034605bb875abd1a0775ec', 'request_id': 'req-348e26da-5e3e-436e-947d-c81a91404d70'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:08.603939 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-348e26da-5e3e-436e-947d-c81a91404d70 admin admin] The token cache is being invalidate because project 59bb100f7d034605bb875abd1a0775ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 02:07:08.604255 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-348e26da-5e3e-436e-947d-c81a91404d70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:08.604954 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-348e26da-5e3e-436e-947d-c81a91404d70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:08.606548 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1376/2781] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 02:07:08 2022] DELETE /identity/v3/projects/59bb100f7d034605bb875abd1a0775ec => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:07:08.610674 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb22343-6777-47d6-a0bf-84fcab0e0a3c admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:08.612306 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-fcb22343-6777-47d6-a0bf-84fcab0e0a3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:07:08.626437 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb22343-6777-47d6-a0bf-84fcab0e0a3c admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:08.687628 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfee33a3-346a-46cc-9f8c-1f495bb2399d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:08.688526 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee33a3-346a-46cc-9f8c-1f495bb2399d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:08.688789 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee33a3-346a-46cc-9f8c-1f495bb2399d admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:08.689058 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfee33a3-346a-46cc-9f8c-1f495bb2399d admin admin] PATH_INFO: `/v3/users/c5a61d0f16044d15b3a26cfa30a7f2fc` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:08.690326 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfee33a3-346a-46cc-9f8c-1f495bb2399d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5a61d0f16044d15b3a26cfa30a7f2fc)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:08.692354 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfee33a3-346a-46cc-9f8c-1f495bb2399d admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:08.698169 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:07:08.743617 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-cfee33a3-346a-46cc-9f8c-1f495bb2399d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5a61d0f16044d15b3a26cfa30a7f2fc'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:08.755818 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-cfee33a3-346a-46cc-9f8c-1f495bb2399d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5a61d0f16044d15b3a26cfa30a7f2fc'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:08.764632 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-cfee33a3-346a-46cc-9f8c-1f495bb2399d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5a61d0f16044d15b3a26cfa30a7f2fc'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:08.773628 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-cfee33a3-346a-46cc-9f8c-1f495bb2399d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5a61d0f16044d15b3a26cfa30a7f2fc'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:08.774218 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-cfee33a3-346a-46cc-9f8c-1f495bb2399d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5a61d0f16044d15b3a26cfa30a7f2fc'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:08.783344 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1406/2782] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 02:07:08 2022] DELETE /identity/v3/users/c5a61d0f16044d15b3a26cfa30a7f2fc => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:07:08.964823 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e26da-5e3e-436e-947d-c81a91404d70 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:08.966376 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-348e26da-5e3e-436e-947d-c81a91404d70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:07:08.981169 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e26da-5e3e-436e-947d-c81a91404d70 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:09.042091 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614f180a-5c54-430e-bf67-b10d21b116e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:09.043050 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f180a-5c54-430e-bf67-b10d21b116e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:09.043658 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f180a-5c54-430e-bf67-b10d21b116e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:09.043954 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614f180a-5c54-430e-bf67-b10d21b116e3 admin admin] PATH_INFO: `/v3/projects/4a08609fee414568a1ead282fbddce87` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:09.045227 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614f180a-5c54-430e-bf67-b10d21b116e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a08609fee414568a1ead282fbddce87)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:09.047195 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614f180a-5c54-430e-bf67-b10d21b116e3 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:09.068239 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:07:09.094783 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-614f180a-5c54-430e-bf67-b10d21b116e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a08609fee414568a1ead282fbddce87', 'request_id': 'req-614f180a-5c54-430e-bf67-b10d21b116e3'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:09.100937 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-614f180a-5c54-430e-bf67-b10d21b116e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a08609fee414568a1ead282fbddce87', 'request_id': 'req-614f180a-5c54-430e-bf67-b10d21b116e3'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:09.108927 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-614f180a-5c54-430e-bf67-b10d21b116e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a08609fee414568a1ead282fbddce87', 'request_id': 'req-614f180a-5c54-430e-bf67-b10d21b116e3'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:09.119541 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-614f180a-5c54-430e-bf67-b10d21b116e3 admin admin] The token cache is being invalidate because project 4a08609fee414568a1ead282fbddce87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 02:07:09.119857 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-614f180a-5c54-430e-bf67-b10d21b116e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:09.120589 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-614f180a-5c54-430e-bf67-b10d21b116e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:09.122174 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1377/2783] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 02:07:08 2022] DELETE /identity/v3/projects/4a08609fee414568a1ead282fbddce87 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:07:18.777187 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:07:19.116466 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:07:29.568606 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfee33a3-346a-46cc-9f8c-1f495bb2399d admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:29.570143 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-cfee33a3-346a-46cc-9f8c-1f495bb2399d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:07:29.672093 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfee33a3-346a-46cc-9f8c-1f495bb2399d admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:29.761431 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e01fcd-beee-43d5-b20f-e5419da39eb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4e83df33b64a3ebf28f54e2d94fe63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:29.762089 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e01fcd-beee-43d5-b20f-e5419da39eb0 service nova] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:29.762372 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e01fcd-beee-43d5-b20f-e5419da39eb0 service nova] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:29.762859 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e01fcd-beee-43d5-b20f-e5419da39eb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:29.764704 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-f5e01fcd-beee-43d5-b20f-e5419da39eb0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:07:29.780739 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e01fcd-beee-43d5-b20f-e5419da39eb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:29.782605 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e01fcd-beee-43d5-b20f-e5419da39eb0 service nova] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:29.819846 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1407/2784] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:07:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:07:33.918352 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614f180a-5c54-430e-bf67-b10d21b116e3 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:33.919880 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-614f180a-5c54-430e-bf67-b10d21b116e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:07:33.994479 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614f180a-5c54-430e-bf67-b10d21b116e3 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:33.999175 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d66e7-89bb-47b0-8646-ec7414cacca3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:33.999864 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d66e7-89bb-47b0-8646-ec7414cacca3 service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:34.000148 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d66e7-89bb-47b0-8646-ec7414cacca3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:34.000422 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d66e7-89bb-47b0-8646-ec7414cacca3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:34.002366 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-5e8d66e7-89bb-47b0-8646-ec7414cacca3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:07:34.017195 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8d66e7-89bb-47b0-8646-ec7414cacca3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:34.018120 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8d66e7-89bb-47b0-8646-ec7414cacca3 service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:34.026857 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1378/2785] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:07:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:07:37.312235 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e01fcd-beee-43d5-b20f-e5419da39eb0 service nova] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:37.322913 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e01fcd-beee-43d5-b20f-e5419da39eb0 service nova] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:37.329407 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0519b-a3eb-4c43-a898-d95d820dfc11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:37.330185 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0519b-a3eb-4c43-a898-d95d820dfc11 service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:37.330563 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0519b-a3eb-4c43-a898-d95d820dfc11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:37.330942 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0519b-a3eb-4c43-a898-d95d820dfc11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:37.332994 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-fbd0519b-a3eb-4c43-a898-d95d820dfc11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:07:37.344420 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd0519b-a3eb-4c43-a898-d95d820dfc11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:37.346224 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd0519b-a3eb-4c43-a898-d95d820dfc11 service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:37.361870 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1408/2786] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:07:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:07:37.476879 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d66e7-89bb-47b0-8646-ec7414cacca3 service neutron] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:37.478325 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-5e8d66e7-89bb-47b0-8646-ec7414cacca3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:07:37.488955 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d66e7-89bb-47b0-8646-ec7414cacca3 service neutron] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:37.493986 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b6d654-1824-4e29-a14f-97d310167fdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:37.494630 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b6d654-1824-4e29-a14f-97d310167fdc service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:37.494912 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b6d654-1824-4e29-a14f-97d310167fdc service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:37.495182 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b6d654-1824-4e29-a14f-97d310167fdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:37.496951 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-52b6d654-1824-4e29-a14f-97d310167fdc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:07:37.510585 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b6d654-1824-4e29-a14f-97d310167fdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:37.511987 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b6d654-1824-4e29-a14f-97d310167fdc service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:37.523505 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1379/2787] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:07:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:07:37.583969 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0519b-a3eb-4c43-a898-d95d820dfc11 service neutron] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:37.585430 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-fbd0519b-a3eb-4c43-a898-d95d820dfc11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:07:37.662967 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0519b-a3eb-4c43-a898-d95d820dfc11 service neutron] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:37.669240 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89bc461-d2d8-4a2e-8219-23eea7f33ac2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5bf3baee1b843fa89b65e5485f6bebb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:37.670119 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89bc461-d2d8-4a2e-8219-23eea7f33ac2 service placement] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:37.670399 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89bc461-d2d8-4a2e-8219-23eea7f33ac2 service placement] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:37.670671 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89bc461-d2d8-4a2e-8219-23eea7f33ac2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:37.672546 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-b89bc461-d2d8-4a2e-8219-23eea7f33ac2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:07:37.684902 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89bc461-d2d8-4a2e-8219-23eea7f33ac2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:37.686515 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89bc461-d2d8-4a2e-8219-23eea7f33ac2 service placement] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:37.692087 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=100 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:07:37.697597 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1409/2788] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:07:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:07:38.449835 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b6d654-1824-4e29-a14f-97d310167fdc service neutron] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:38.457191 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-52b6d654-1824-4e29-a14f-97d310167fdc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:07:38.487721 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b6d654-1824-4e29-a14f-97d310167fdc service neutron] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:38.501740 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb292c3-ede1-4755-be46-e197ad7ce757 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5bf3baee1b843fa89b65e5485f6bebb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:38.502822 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb292c3-ede1-4755-be46-e197ad7ce757 service placement] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:38.503434 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb292c3-ede1-4755-be46-e197ad7ce757 service placement] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:38.504208 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb292c3-ede1-4755-be46-e197ad7ce757 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:38.511427 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-9cb292c3-ede1-4755-be46-e197ad7ce757 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:07:38.512598 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89bc461-d2d8-4a2e-8219-23eea7f33ac2 service placement] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:38.514046 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-b89bc461-d2d8-4a2e-8219-23eea7f33ac2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:07:38.531771 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb292c3-ede1-4755-be46-e197ad7ce757 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:38.533849 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb292c3-ede1-4755-be46-e197ad7ce757 service placement] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:38.570013 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1380/2789] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:07:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:07:38.634617 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89bc461-d2d8-4a2e-8219-23eea7f33ac2 service placement] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:38.643470 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0965c851-135d-4407-b361-9bb3724ddfcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:38.644361 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965c851-135d-4407-b361-9bb3724ddfcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:38.644770 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965c851-135d-4407-b361-9bb3724ddfcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:38.645045 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965c851-135d-4407-b361-9bb3724ddfcb admin admin] PATH_INFO: `/v3/users/ffe72dd809a84bd5aae15a85f081324e` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:38.646258 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0965c851-135d-4407-b361-9bb3724ddfcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffe72dd809a84bd5aae15a85f081324e)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:38.648416 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0965c851-135d-4407-b361-9bb3724ddfcb admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:38.699167 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-0965c851-135d-4407-b361-9bb3724ddfcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffe72dd809a84bd5aae15a85f081324e'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:38.716974 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-0965c851-135d-4407-b361-9bb3724ddfcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffe72dd809a84bd5aae15a85f081324e'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:38.725996 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-0965c851-135d-4407-b361-9bb3724ddfcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffe72dd809a84bd5aae15a85f081324e'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:38.733496 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-0965c851-135d-4407-b361-9bb3724ddfcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffe72dd809a84bd5aae15a85f081324e'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:38.734137 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-0965c851-135d-4407-b361-9bb3724ddfcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffe72dd809a84bd5aae15a85f081324e'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:38.740115 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1410/2790] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 02:07:38 2022] DELETE /identity/v3/users/ffe72dd809a84bd5aae15a85f081324e => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 02:07:38.920851 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb292c3-ede1-4755-be46-e197ad7ce757 service placement] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:38.932988 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb292c3-ede1-4755-be46-e197ad7ce757 service placement] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:38.995093 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dcc0b7-3b40-4fea-a9e8-f4d7f63f2b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:38.996046 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dcc0b7-3b40-4fea-a9e8-f4d7f63f2b27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:38.996333 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dcc0b7-3b40-4fea-a9e8-f4d7f63f2b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:38.996634 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dcc0b7-3b40-4fea-a9e8-f4d7f63f2b27 admin admin] PATH_INFO: `/v3/projects/07da57919f294173b2dc00509c4a6e0f` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:38.998065 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dcc0b7-3b40-4fea-a9e8-f4d7f63f2b27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07da57919f294173b2dc00509c4a6e0f)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:39.000917 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dcc0b7-3b40-4fea-a9e8-f4d7f63f2b27 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:39.061093 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-26dcc0b7-3b40-4fea-a9e8-f4d7f63f2b27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07da57919f294173b2dc00509c4a6e0f', 'request_id': 'req-26dcc0b7-3b40-4fea-a9e8-f4d7f63f2b27'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:39.068171 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-26dcc0b7-3b40-4fea-a9e8-f4d7f63f2b27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07da57919f294173b2dc00509c4a6e0f', 'request_id': 'req-26dcc0b7-3b40-4fea-a9e8-f4d7f63f2b27'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:39.077526 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-26dcc0b7-3b40-4fea-a9e8-f4d7f63f2b27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07da57919f294173b2dc00509c4a6e0f', 'request_id': 'req-26dcc0b7-3b40-4fea-a9e8-f4d7f63f2b27'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:39.089311 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-26dcc0b7-3b40-4fea-a9e8-f4d7f63f2b27 admin admin] The token cache is being invalidate because project 07da57919f294173b2dc00509c4a6e0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 02:07:39.089703 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-26dcc0b7-3b40-4fea-a9e8-f4d7f63f2b27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:39.090373 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-26dcc0b7-3b40-4fea-a9e8-f4d7f63f2b27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:39.092010 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1381/2791] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 02:07:38 2022] DELETE /identity/v3/projects/07da57919f294173b2dc00509c4a6e0f => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:07:39.097125 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b7f7b-065b-4d5e-b757-3cdacb6b5508 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:39.097497 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b7f7b-065b-4d5e-b757-3cdacb6b5508 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:39.097841 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b7f7b-065b-4d5e-b757-3cdacb6b5508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:39.118851 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-fd4b7f7b-065b-4d5e-b757-3cdacb6b5508 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:07:39.120238 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-fd4b7f7b-065b-4d5e-b757-3cdacb6b5508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:07:39.182092 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1411/2792] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:07:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:07:39.190072 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884f96e-3035-45e5-adfb-b4bdecb94433 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:39.190358 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884f96e-3035-45e5-adfb-b4bdecb94433 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:39.190685 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884f96e-3035-45e5-adfb-b4bdecb94433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:39.203990 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:07:39.210678 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-4884f96e-3035-45e5-adfb-b4bdecb94433 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:07:39.212821 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-4884f96e-3035-45e5-adfb-b4bdecb94433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:07:39.221643 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1382/2793] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:07:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:07:39.225874 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b7f7b-065b-4d5e-b757-3cdacb6b5508 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:39.235881 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b7f7b-065b-4d5e-b757-3cdacb6b5508 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:39.241078 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5eba24-e14e-4404-babc-60eecdc368c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:39.241760 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5eba24-e14e-4404-babc-60eecdc368c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:39.242038 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5eba24-e14e-4404-babc-60eecdc368c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:39.242446 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5eba24-e14e-4404-babc-60eecdc368c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:39.243187 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5eba24-e14e-4404-babc-60eecdc368c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 02:07:39.243527 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5eba24-e14e-4404-babc-60eecdc368c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:39.245149 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5eba24-e14e-4404-babc-60eecdc368c8 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:39.256890 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1412/2794] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 02:07:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 02:07:39.261061 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4884f96e-3035-45e5-adfb-b4bdecb94433 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:39.270798 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4884f96e-3035-45e5-adfb-b4bdecb94433 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:39.275743 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89537967-1f60-485b-b0a6-ee21201a82c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:39.276403 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89537967-1f60-485b-b0a6-ee21201a82c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:39.276686 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89537967-1f60-485b-b0a6-ee21201a82c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:39.276962 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89537967-1f60-485b-b0a6-ee21201a82c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:39.277669 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89537967-1f60-485b-b0a6-ee21201a82c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:39.279681 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89537967-1f60-485b-b0a6-ee21201a82c2 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:39.308120 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1383/2795] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 02:07:39 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:07:39.312687 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5eba24-e14e-4404-babc-60eecdc368c8 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:39.323467 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5eba24-e14e-4404-babc-60eecdc368c8 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:39.327365 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:07:39.380701 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d688e55f-23dc-449b-96fb-3a0e5d131470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:39.381373 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688e55f-23dc-449b-96fb-3a0e5d131470 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:39.381720 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688e55f-23dc-449b-96fb-3a0e5d131470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:39.382060 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688e55f-23dc-449b-96fb-3a0e5d131470 admin admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:39.382806 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d688e55f-23dc-449b-96fb-3a0e5d131470 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:39.384814 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d688e55f-23dc-449b-96fb-3a0e5d131470 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:39.408979 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1413/2796] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 02:07:39 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:07:39.410197 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89537967-1f60-485b-b0a6-ee21201a82c2 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:39.421162 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89537967-1f60-485b-b0a6-ee21201a82c2 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:39.425707 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69156db6-38b9-428d-9b0d-d1d8a5f19e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:39.426475 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69156db6-38b9-428d-9b0d-d1d8a5f19e43 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:39.426717 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69156db6-38b9-428d-9b0d-d1d8a5f19e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:39.427024 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69156db6-38b9-428d-9b0d-d1d8a5f19e43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:39.427785 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69156db6-38b9-428d-9b0d-d1d8a5f19e43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:39.429378 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69156db6-38b9-428d-9b0d-d1d8a5f19e43 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:39.443404 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1384/2797] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:07:39.444749 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d688e55f-23dc-449b-96fb-3a0e5d131470 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:39.455487 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d688e55f-23dc-449b-96fb-3a0e5d131470 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:39.462069 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54973969-1b8d-4d31-acd6-b11dbf69d932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:39.462738 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54973969-1b8d-4d31-acd6-b11dbf69d932 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:39.463022 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54973969-1b8d-4d31-acd6-b11dbf69d932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:39.463347 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54973969-1b8d-4d31-acd6-b11dbf69d932 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:39.464088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54973969-1b8d-4d31-acd6-b11dbf69d932 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:39.465706 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54973969-1b8d-4d31-acd6-b11dbf69d932 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:39.476299 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1414/2798] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:07:39.480499 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69156db6-38b9-428d-9b0d-d1d8a5f19e43 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:39.491056 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69156db6-38b9-428d-9b0d-d1d8a5f19e43 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:39.496776 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39998381-d9bf-4e28-a4c8-6345ad19b333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:39.497671 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39998381-d9bf-4e28-a4c8-6345ad19b333 admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:39.497947 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39998381-d9bf-4e28-a4c8-6345ad19b333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:39.498219 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39998381-d9bf-4e28-a4c8-6345ad19b333 admin admin] PATH_INFO: `/v3/projects/0ba4afb05fe54c9885b41cb4d6a3bfda/users/1a39cf5ed28a4988a9aef27ac6677cda/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:39.516123 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39998381-d9bf-4e28-a4c8-6345ad19b333 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ba4afb05fe54c9885b41cb4d6a3bfda, user_id=1a39cf5ed28a4988a9aef27ac6677cda, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:39.518808 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39998381-d9bf-4e28-a4c8-6345ad19b333 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:39.532878 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1385/2799] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 02:07:39 2022] PUT /identity/v3/projects/0ba4afb05fe54c9885b41cb4d6a3bfda/users/1a39cf5ed28a4988a9aef27ac6677cda/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:07:39.542785 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df972363-f2b6-41ce-a43b-325c4e5bf3bf None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:39.543313 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df972363-f2b6-41ce-a43b-325c4e5bf3bf None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:39.543419 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df972363-f2b6-41ce-a43b-325c4e5bf3bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:39.569621 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-df972363-f2b6-41ce-a43b-325c4e5bf3bf None None] MFA Rules not processed for user `1a39cf5ed28a4988a9aef27ac6677cda`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:07:39.570852 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-df972363-f2b6-41ce-a43b-325c4e5bf3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:07:39.646331 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1415/2800] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:07:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:07:39.650778 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39998381-d9bf-4e28-a4c8-6345ad19b333 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:39.716866 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39998381-d9bf-4e28-a4c8-6345ad19b333 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:39.721933 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28f1ffb-1f3f-4254-94e1-819b8592b8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:39.722596 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f1ffb-1f3f-4254-94e1-819b8592b8e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:39.722874 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f1ffb-1f3f-4254-94e1-819b8592b8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:39.723149 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f1ffb-1f3f-4254-94e1-819b8592b8e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:39.723888 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28f1ffb-1f3f-4254-94e1-819b8592b8e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:39.725783 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28f1ffb-1f3f-4254-94e1-819b8592b8e6 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:39.753772 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1386/2801] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 02:07:39 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:07:39.757925 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df972363-f2b6-41ce-a43b-325c4e5bf3bf None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:39.769247 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df972363-f2b6-41ce-a43b-325c4e5bf3bf None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:39.827663 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05463e54-ad69-4392-9510-7120a72d5fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:39.828333 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05463e54-ad69-4392-9510-7120a72d5fd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:39.828609 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05463e54-ad69-4392-9510-7120a72d5fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:39.828876 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05463e54-ad69-4392-9510-7120a72d5fd1 admin admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:39.829560 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05463e54-ad69-4392-9510-7120a72d5fd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:39.831478 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05463e54-ad69-4392-9510-7120a72d5fd1 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:39.854006 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1416/2802] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 02:07:39 2022] POST /identity/v3/users => generated 299 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:07:39.855269 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28f1ffb-1f3f-4254-94e1-819b8592b8e6 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:39.865876 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28f1ffb-1f3f-4254-94e1-819b8592b8e6 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:39.871064 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea0243a-85fe-43c7-bc01-a4833faf0316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:39.871750 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea0243a-85fe-43c7-bc01-a4833faf0316 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:39.872098 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea0243a-85fe-43c7-bc01-a4833faf0316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:39.872486 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea0243a-85fe-43c7-bc01-a4833faf0316 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:39.873539 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea0243a-85fe-43c7-bc01-a4833faf0316 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:39.875734 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea0243a-85fe-43c7-bc01-a4833faf0316 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:39.886352 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1387/2803] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:07:39.890575 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05463e54-ad69-4392-9510-7120a72d5fd1 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:39.897223 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:07:39.901208 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05463e54-ad69-4392-9510-7120a72d5fd1 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:39.907100 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66889e70-0ba8-4fbb-98e5-8f26ead7f7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:39.908014 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66889e70-0ba8-4fbb-98e5-8f26ead7f7ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:39.908293 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66889e70-0ba8-4fbb-98e5-8f26ead7f7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:39.908570 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66889e70-0ba8-4fbb-98e5-8f26ead7f7ea admin admin] PATH_INFO: `/v3/projects/85ecf6bc6fe440158c82c12f401fda53/users/bd713cd593b449b1808c32390df0a8c7/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:39.925403 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66889e70-0ba8-4fbb-98e5-8f26ead7f7ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=85ecf6bc6fe440158c82c12f401fda53, user_id=bd713cd593b449b1808c32390df0a8c7, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:39.928111 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66889e70-0ba8-4fbb-98e5-8f26ead7f7ea admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:39.949983 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1417/2804] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 02:07:39 2022] PUT /identity/v3/projects/85ecf6bc6fe440158c82c12f401fda53/users/bd713cd593b449b1808c32390df0a8c7/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:07:39.954097 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea0243a-85fe-43c7-bc01-a4833faf0316 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:39.964588 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea0243a-85fe-43c7-bc01-a4833faf0316 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:40.033623 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b560f05-68c8-4a6a-a866-a5bfbaa9c4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:40.034296 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b560f05-68c8-4a6a-a866-a5bfbaa9c4db admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:40.034572 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b560f05-68c8-4a6a-a866-a5bfbaa9c4db admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:40.034839 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b560f05-68c8-4a6a-a866-a5bfbaa9c4db admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:40.035598 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b560f05-68c8-4a6a-a866-a5bfbaa9c4db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:40.037194 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b560f05-68c8-4a6a-a866-a5bfbaa9c4db admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:40.048007 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1388/2805] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:07:40.052361 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66889e70-0ba8-4fbb-98e5-8f26ead7f7ea admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:40.063177 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66889e70-0ba8-4fbb-98e5-8f26ead7f7ea admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:40.064501 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b560f05-68c8-4a6a-a866-a5bfbaa9c4db admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:40.067370 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-8b560f05-68c8-4a6a-a866-a5bfbaa9c4db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:07:40.071866 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef055ee-4df0-4010-a5ec-aa5d4ae17b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:40.072774 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef055ee-4df0-4010-a5ec-aa5d4ae17b08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:40.073054 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef055ee-4df0-4010-a5ec-aa5d4ae17b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:40.073347 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef055ee-4df0-4010-a5ec-aa5d4ae17b08 admin admin] PATH_INFO: `/v3/projects/85ecf6bc6fe440158c82c12f401fda53/users/bd713cd593b449b1808c32390df0a8c7/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:40.075450 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef055ee-4df0-4010-a5ec-aa5d4ae17b08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85ecf6bc6fe440158c82c12f401fda53, user_id=bd713cd593b449b1808c32390df0a8c7, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:40.078175 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef055ee-4df0-4010-a5ec-aa5d4ae17b08 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:40.085782 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:07:40.094187 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1418/2806] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 02:07:40 2022] PUT /identity/v3/projects/85ecf6bc6fe440158c82c12f401fda53/users/bd713cd593b449b1808c32390df0a8c7/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 02:07:40.101372 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee820818-e4eb-4199-b97e-802501ac2f46 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:40.101874 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee820818-e4eb-4199-b97e-802501ac2f46 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:40.102355 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee820818-e4eb-4199-b97e-802501ac2f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:40.131770 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-ee820818-e4eb-4199-b97e-802501ac2f46 None None] MFA Rules not processed for user `bd713cd593b449b1808c32390df0a8c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:07:40.133146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-ee820818-e4eb-4199-b97e-802501ac2f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:07:40.165993 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b560f05-68c8-4a6a-a866-a5bfbaa9c4db admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:40.208382 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1419/2807] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:07:40 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:07:40.217028 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee820818-e4eb-4199-b97e-802501ac2f46 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:40.218504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-ee820818-e4eb-4199-b97e-802501ac2f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:07:40.238352 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d0b3da-fbbf-49aa-92b1-ad440f6d2a1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae4e83df33b64a3ebf28f54e2d94fe63', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:40.239450 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d0b3da-fbbf-49aa-92b1-ad440f6d2a1e service nova] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:40.239828 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d0b3da-fbbf-49aa-92b1-ad440f6d2a1e service nova] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:40.241583 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d0b3da-fbbf-49aa-92b1-ad440f6d2a1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:40.243735 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-a5d0b3da-fbbf-49aa-92b1-ad440f6d2a1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:07:40.256693 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d0b3da-fbbf-49aa-92b1-ad440f6d2a1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:40.258291 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d0b3da-fbbf-49aa-92b1-ad440f6d2a1e service nova] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:40.271010 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1389/2808] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:07:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 02:07:40.283411 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee820818-e4eb-4199-b97e-802501ac2f46 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:40.287937 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c209a01b-3ac0-47d9-889b-09ac42c4d2af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:40.288713 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209a01b-3ac0-47d9-889b-09ac42c4d2af service cinder] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:40.289548 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209a01b-3ac0-47d9-889b-09ac42c4d2af service cinder] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:40.289833 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209a01b-3ac0-47d9-889b-09ac42c4d2af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:40.303080 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c209a01b-3ac0-47d9-889b-09ac42c4d2af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:40.304774 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c209a01b-3ac0-47d9-889b-09ac42c4d2af service cinder] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:40.315236 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1420/2809] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:07:40 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 02:07:40.344881 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d0b3da-fbbf-49aa-92b1-ad440f6d2a1e service nova] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:40.408881 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d0b3da-fbbf-49aa-92b1-ad440f6d2a1e service nova] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:40.414869 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263dc575-5769-4160-a10c-9c58a3365f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:40.415780 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263dc575-5769-4160-a10c-9c58a3365f6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:40.416137 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263dc575-5769-4160-a10c-9c58a3365f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:40.416307 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263dc575-5769-4160-a10c-9c58a3365f6b admin admin] PATH_INFO: `/v3/users/1a39cf5ed28a4988a9aef27ac6677cda` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:40.417689 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263dc575-5769-4160-a10c-9c58a3365f6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a39cf5ed28a4988a9aef27ac6677cda)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:40.420210 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263dc575-5769-4160-a10c-9c58a3365f6b admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:40.465573 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:07:40.472522 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-263dc575-5769-4160-a10c-9c58a3365f6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a39cf5ed28a4988a9aef27ac6677cda'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:40.483871 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-263dc575-5769-4160-a10c-9c58a3365f6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a39cf5ed28a4988a9aef27ac6677cda'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:40.493122 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-263dc575-5769-4160-a10c-9c58a3365f6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a39cf5ed28a4988a9aef27ac6677cda'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:40.501953 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-263dc575-5769-4160-a10c-9c58a3365f6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a39cf5ed28a4988a9aef27ac6677cda'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:40.502729 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-263dc575-5769-4160-a10c-9c58a3365f6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a39cf5ed28a4988a9aef27ac6677cda'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:40.510223 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1390/2810] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 02:07:40 2022] DELETE /identity/v3/users/1a39cf5ed28a4988a9aef27ac6677cda => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:07:40.514916 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891b8eb-4d50-4ed5-86a5-a2ba26483b5e None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:40.515200 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891b8eb-4d50-4ed5-86a5-a2ba26483b5e None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:40.515500 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891b8eb-4d50-4ed5-86a5-a2ba26483b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:40.533830 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-3891b8eb-4d50-4ed5-86a5-a2ba26483b5e None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:07:40.535035 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-3891b8eb-4d50-4ed5-86a5-a2ba26483b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:07:40.585530 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:07:40.598941 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1421/2811] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:07:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:07:40.608815 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263dc575-5769-4160-a10c-9c58a3365f6b admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:40.610496 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-263dc575-5769-4160-a10c-9c58a3365f6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:07:40.673182 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263dc575-5769-4160-a10c-9c58a3365f6b admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:40.677223 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae9842-c01b-4f91-a0c2-3679d6ddd36b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:40.677906 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae9842-c01b-4f91-a0c2-3679d6ddd36b service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:40.678184 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae9842-c01b-4f91-a0c2-3679d6ddd36b service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:40.678449 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae9842-c01b-4f91-a0c2-3679d6ddd36b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:40.687830 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae9842-c01b-4f91-a0c2-3679d6ddd36b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:40.689270 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae9842-c01b-4f91-a0c2-3679d6ddd36b service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:40.699061 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1391/2812] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:07:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:07:40.986203 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3891b8eb-4d50-4ed5-86a5-a2ba26483b5e None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:40.987824 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-3891b8eb-4d50-4ed5-86a5-a2ba26483b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:07:41.002469 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3891b8eb-4d50-4ed5-86a5-a2ba26483b5e None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:41.015868 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a3d5b-3abc-4d58-9a7c-61e8bb040e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:41.015868 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a3d5b-3abc-4d58-9a7c-61e8bb040e41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:41.015868 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a3d5b-3abc-4d58-9a7c-61e8bb040e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:41.015868 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a3d5b-3abc-4d58-9a7c-61e8bb040e41 admin admin] PATH_INFO: `/v3/projects/0ba4afb05fe54c9885b41cb4d6a3bfda` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:41.016548 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6a3d5b-3abc-4d58-9a7c-61e8bb040e41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ba4afb05fe54c9885b41cb4d6a3bfda)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:41.016548 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6a3d5b-3abc-4d58-9a7c-61e8bb040e41 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:41.064077 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-1b6a3d5b-3abc-4d58-9a7c-61e8bb040e41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ba4afb05fe54c9885b41cb4d6a3bfda', 'request_id': 'req-1b6a3d5b-3abc-4d58-9a7c-61e8bb040e41'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:41.070048 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-1b6a3d5b-3abc-4d58-9a7c-61e8bb040e41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ba4afb05fe54c9885b41cb4d6a3bfda', 'request_id': 'req-1b6a3d5b-3abc-4d58-9a7c-61e8bb040e41'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:41.078380 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-1b6a3d5b-3abc-4d58-9a7c-61e8bb040e41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ba4afb05fe54c9885b41cb4d6a3bfda', 'request_id': 'req-1b6a3d5b-3abc-4d58-9a7c-61e8bb040e41'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:41.089896 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-1b6a3d5b-3abc-4d58-9a7c-61e8bb040e41 admin admin] The token cache is being invalidate because project 0ba4afb05fe54c9885b41cb4d6a3bfda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 02:07:41.090312 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-1b6a3d5b-3abc-4d58-9a7c-61e8bb040e41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:41.091362 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-1b6a3d5b-3abc-4d58-9a7c-61e8bb040e41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:41.093767 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1422/2813] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 02:07:40 2022] DELETE /identity/v3/projects/0ba4afb05fe54c9885b41cb4d6a3bfda => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:07:41.098003 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae9842-c01b-4f91-a0c2-3679d6ddd36b service neutron] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:41.107992 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae9842-c01b-4f91-a0c2-3679d6ddd36b service neutron] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:41.171743 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a85923-9c1c-4e46-9707-197e9cfbd644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:41.172646 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a85923-9c1c-4e46-9707-197e9cfbd644 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:41.172921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a85923-9c1c-4e46-9707-197e9cfbd644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:41.173267 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a85923-9c1c-4e46-9707-197e9cfbd644 admin admin] PATH_INFO: `/v3/users/bd713cd593b449b1808c32390df0a8c7` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:41.174663 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a85923-9c1c-4e46-9707-197e9cfbd644 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd713cd593b449b1808c32390df0a8c7)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:41.176699 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a85923-9c1c-4e46-9707-197e9cfbd644 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:41.225338 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-16a85923-9c1c-4e46-9707-197e9cfbd644 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd713cd593b449b1808c32390df0a8c7'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:41.238900 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:07:41.242656 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-16a85923-9c1c-4e46-9707-197e9cfbd644 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd713cd593b449b1808c32390df0a8c7'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:41.257107 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-16a85923-9c1c-4e46-9707-197e9cfbd644 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd713cd593b449b1808c32390df0a8c7'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:41.270447 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-16a85923-9c1c-4e46-9707-197e9cfbd644 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd713cd593b449b1808c32390df0a8c7'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:41.271634 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-16a85923-9c1c-4e46-9707-197e9cfbd644 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd713cd593b449b1808c32390df0a8c7'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:41.280850 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1392/2814] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 02:07:41 2022] DELETE /identity/v3/users/bd713cd593b449b1808c32390df0a8c7 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:07:41.564937 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a3d5b-3abc-4d58-9a7c-61e8bb040e41 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:41.566533 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-1b6a3d5b-3abc-4d58-9a7c-61e8bb040e41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:07:41.580203 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a3d5b-3abc-4d58-9a7c-61e8bb040e41 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:41.643481 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac2b161-8658-4f86-8c0c-270f216f71b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:41.644343 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2b161-8658-4f86-8c0c-270f216f71b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:41.644620 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2b161-8658-4f86-8c0c-270f216f71b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:41.644891 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2b161-8658-4f86-8c0c-270f216f71b1 admin admin] PATH_INFO: `/v3/projects/85ecf6bc6fe440158c82c12f401fda53` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:41.646073 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac2b161-8658-4f86-8c0c-270f216f71b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85ecf6bc6fe440158c82c12f401fda53)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:41.648013 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac2b161-8658-4f86-8c0c-270f216f71b1 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:41.696299 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-9ac2b161-8658-4f86-8c0c-270f216f71b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85ecf6bc6fe440158c82c12f401fda53', 'request_id': 'req-9ac2b161-8658-4f86-8c0c-270f216f71b1'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:41.703935 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-9ac2b161-8658-4f86-8c0c-270f216f71b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85ecf6bc6fe440158c82c12f401fda53', 'request_id': 'req-9ac2b161-8658-4f86-8c0c-270f216f71b1'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:41.711850 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-9ac2b161-8658-4f86-8c0c-270f216f71b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85ecf6bc6fe440158c82c12f401fda53', 'request_id': 'req-9ac2b161-8658-4f86-8c0c-270f216f71b1'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:41.722735 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-9ac2b161-8658-4f86-8c0c-270f216f71b1 admin admin] The token cache is being invalidate because project 85ecf6bc6fe440158c82c12f401fda53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 02:07:41.723010 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-9ac2b161-8658-4f86-8c0c-270f216f71b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:41.723678 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-9ac2b161-8658-4f86-8c0c-270f216f71b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:41.725222 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1423/2815] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 02:07:41 2022] DELETE /identity/v3/projects/85ecf6bc6fe440158c82c12f401fda53 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 02:07:41.730597 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8f964-be68-43a8-8af6-e5ccbc0f941c None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:41.731132 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8f964-be68-43a8-8af6-e5ccbc0f941c None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:41.731132 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8f964-be68-43a8-8af6-e5ccbc0f941c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:41.749731 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-3ce8f964-be68-43a8-8af6-e5ccbc0f941c None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:07:41.750864 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-3ce8f964-be68-43a8-8af6-e5ccbc0f941c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:07:41.810966 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1393/2816] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:07:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 02:07:41.818583 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e91d2-87e2-4a1c-9936-0955ff1c2913 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:41.818989 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e91d2-87e2-4a1c-9936-0955ff1c2913 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:41.819131 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e91d2-87e2-4a1c-9936-0955ff1c2913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:41.838801 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-538e91d2-87e2-4a1c-9936-0955ff1c2913 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:07:41.840023 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-538e91d2-87e2-4a1c-9936-0955ff1c2913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:07:41.848219 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1424/2817] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:07:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:07:41.852446 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8f964-be68-43a8-8af6-e5ccbc0f941c None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:41.862653 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8f964-be68-43a8-8af6-e5ccbc0f941c None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:41.868389 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d19414e-24ce-4a44-bfd4-f3c09edfeafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:41.869073 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d19414e-24ce-4a44-bfd4-f3c09edfeafe admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:41.869354 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d19414e-24ce-4a44-bfd4-f3c09edfeafe admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:41.869615 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d19414e-24ce-4a44-bfd4-f3c09edfeafe admin admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:41.870356 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d19414e-24ce-4a44-bfd4-f3c09edfeafe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 02:07:41.870668 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d19414e-24ce-4a44-bfd4-f3c09edfeafe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:41.872363 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d19414e-24ce-4a44-bfd4-f3c09edfeafe admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:41.884199 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1394/2818] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 02:07:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 02:07:41.888353 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e91d2-87e2-4a1c-9936-0955ff1c2913 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:41.892721 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:07:41.899111 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e91d2-87e2-4a1c-9936-0955ff1c2913 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:41.905833 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b023d318-0802-4f4e-8298-e7c03f51604d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:41.906481 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023d318-0802-4f4e-8298-e7c03f51604d admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:41.906768 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023d318-0802-4f4e-8298-e7c03f51604d admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:41.907040 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023d318-0802-4f4e-8298-e7c03f51604d admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:41.907773 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b023d318-0802-4f4e-8298-e7c03f51604d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:41.909718 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b023d318-0802-4f4e-8298-e7c03f51604d admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:41.941840 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1425/2819] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 02:07:41 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:07:41.947181 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d19414e-24ce-4a44-bfd4-f3c09edfeafe admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:41.957563 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d19414e-24ce-4a44-bfd4-f3c09edfeafe admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:42.021919 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a9fd35-ba9b-43b4-bd8b-62f21812df9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:42.022515 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a9fd35-ba9b-43b4-bd8b-62f21812df9d admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:42.022798 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a9fd35-ba9b-43b4-bd8b-62f21812df9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:42.023077 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a9fd35-ba9b-43b4-bd8b-62f21812df9d admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:42.023857 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a9fd35-ba9b-43b4-bd8b-62f21812df9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:42.025955 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a9fd35-ba9b-43b4-bd8b-62f21812df9d admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:42.048648 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1395/2820] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 02:07:41 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:07:42.053085 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b023d318-0802-4f4e-8298-e7c03f51604d admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:42.064061 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b023d318-0802-4f4e-8298-e7c03f51604d admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:42.069420 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bcafae-5458-43f8-a838-c6650b6c7b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:42.070088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcafae-5458-43f8-a838-c6650b6c7b00 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:42.070369 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcafae-5458-43f8-a838-c6650b6c7b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:42.070642 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcafae-5458-43f8-a838-c6650b6c7b00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:42.071431 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bcafae-5458-43f8-a838-c6650b6c7b00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:42.073314 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bcafae-5458-43f8-a838-c6650b6c7b00 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:42.085098 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1426/2821] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:07:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:07:42.089233 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a9fd35-ba9b-43b4-bd8b-62f21812df9d admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:42.100021 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a9fd35-ba9b-43b4-bd8b-62f21812df9d admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:42.105206 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11085657-4510-4f42-8e3b-ecb6c393c296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:42.105892 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11085657-4510-4f42-8e3b-ecb6c393c296 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:42.106173 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11085657-4510-4f42-8e3b-ecb6c393c296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:42.106451 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11085657-4510-4f42-8e3b-ecb6c393c296 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:42.107227 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11085657-4510-4f42-8e3b-ecb6c393c296 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:42.109202 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11085657-4510-4f42-8e3b-ecb6c393c296 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:42.127900 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1396/2822] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:07:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:07:42.129135 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bcafae-5458-43f8-a838-c6650b6c7b00 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:42.139271 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bcafae-5458-43f8-a838-c6650b6c7b00 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:42.144030 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b89c12-10a2-4bee-ac62-78b0c8078992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:42.144981 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b89c12-10a2-4bee-ac62-78b0c8078992 admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:42.145319 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b89c12-10a2-4bee-ac62-78b0c8078992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:42.145651 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b89c12-10a2-4bee-ac62-78b0c8078992 admin admin] PATH_INFO: `/v3/projects/6045a976bcf3436dac4c596a10829ed7/users/60a8c17b37844db69bfa9d0befed526d/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:42.161497 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b89c12-10a2-4bee-ac62-78b0c8078992 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6045a976bcf3436dac4c596a10829ed7, user_id=60a8c17b37844db69bfa9d0befed526d, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:42.164314 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b89c12-10a2-4bee-ac62-78b0c8078992 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:42.180228 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1427/2823] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 02:07:42 2022] PUT /identity/v3/projects/6045a976bcf3436dac4c596a10829ed7/users/60a8c17b37844db69bfa9d0befed526d/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:07:42.190681 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8da7e2e-67b1-4ec1-b2f0-b28e047c6c92 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:42.190970 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8da7e2e-67b1-4ec1-b2f0-b28e047c6c92 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:42.191275 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8da7e2e-67b1-4ec1-b2f0-b28e047c6c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:42.218266 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-b8da7e2e-67b1-4ec1-b2f0-b28e047c6c92 None None] MFA Rules not processed for user `60a8c17b37844db69bfa9d0befed526d`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:07:42.219744 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-b8da7e2e-67b1-4ec1-b2f0-b28e047c6c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:07:42.253103 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:07:42.294512 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1397/2824] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:07:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:07:42.299687 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b89c12-10a2-4bee-ac62-78b0c8078992 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:42.365397 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b89c12-10a2-4bee-ac62-78b0c8078992 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:42.370827 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250a5c0a-226c-4ed0-99dc-75f30630c544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:42.371504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a5c0a-226c-4ed0-99dc-75f30630c544 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:42.371791 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a5c0a-226c-4ed0-99dc-75f30630c544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:42.372092 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a5c0a-226c-4ed0-99dc-75f30630c544 admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:42.372774 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250a5c0a-226c-4ed0-99dc-75f30630c544 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:42.374739 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250a5c0a-226c-4ed0-99dc-75f30630c544 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:42.402221 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1428/2825] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 02:07:42 2022] POST /identity/v3/projects => generated 387 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:07:42.406549 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8da7e2e-67b1-4ec1-b2f0-b28e047c6c92 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:42.418057 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8da7e2e-67b1-4ec1-b2f0-b28e047c6c92 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:42.478200 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cdaa19-6a75-4d7f-af1c-709b32aef5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:42.478868 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cdaa19-6a75-4d7f-af1c-709b32aef5cd admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:42.479151 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cdaa19-6a75-4d7f-af1c-709b32aef5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:42.479456 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cdaa19-6a75-4d7f-af1c-709b32aef5cd admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:42.480145 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cdaa19-6a75-4d7f-af1c-709b32aef5cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:42.482230 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cdaa19-6a75-4d7f-af1c-709b32aef5cd admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:42.503067 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1398/2826] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 02:07:42 2022] POST /identity/v3/users => generated 300 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:07:42.507335 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250a5c0a-226c-4ed0-99dc-75f30630c544 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:42.518650 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250a5c0a-226c-4ed0-99dc-75f30630c544 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:42.524394 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f543d29-bdc4-4e38-89de-ba012bc66640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:42.525048 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f543d29-bdc4-4e38-89de-ba012bc66640 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:42.525327 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f543d29-bdc4-4e38-89de-ba012bc66640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:42.525600 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f543d29-bdc4-4e38-89de-ba012bc66640 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:42.526333 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f543d29-bdc4-4e38-89de-ba012bc66640 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:42.527981 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f543d29-bdc4-4e38-89de-ba012bc66640 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:42.539391 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1429/2827] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:07:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:07:42.543842 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cdaa19-6a75-4d7f-af1c-709b32aef5cd admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:42.554887 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cdaa19-6a75-4d7f-af1c-709b32aef5cd admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:42.560750 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f756d7-9c93-48d5-b1ad-41618881e88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:42.561635 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f756d7-9c93-48d5-b1ad-41618881e88b admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:42.561921 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f756d7-9c93-48d5-b1ad-41618881e88b admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:42.562189 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f756d7-9c93-48d5-b1ad-41618881e88b admin admin] PATH_INFO: `/v3/projects/d50e1392921f459aada27b8319a1adb0/users/ee8f674890cb4f078ec87a889c5e4764/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:42.578539 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f756d7-9c93-48d5-b1ad-41618881e88b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d50e1392921f459aada27b8319a1adb0, user_id=ee8f674890cb4f078ec87a889c5e4764, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:42.582268 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f756d7-9c93-48d5-b1ad-41618881e88b admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:42.603679 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1399/2828] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 02:07:42 2022] PUT /identity/v3/projects/d50e1392921f459aada27b8319a1adb0/users/ee8f674890cb4f078ec87a889c5e4764/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:07:42.608972 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f543d29-bdc4-4e38-89de-ba012bc66640 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:42.620211 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f543d29-bdc4-4e38-89de-ba012bc66640 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:42.687774 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec2475-b7ff-41c4-b0c9-4e0bcf7f7e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:42.688425 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec2475-b7ff-41c4-b0c9-4e0bcf7f7e3d admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:42.688696 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec2475-b7ff-41c4-b0c9-4e0bcf7f7e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:42.688971 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec2475-b7ff-41c4-b0c9-4e0bcf7f7e3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:42.689709 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ec2475-b7ff-41c4-b0c9-4e0bcf7f7e3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:42.691359 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ec2475-b7ff-41c4-b0c9-4e0bcf7f7e3d admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:42.701448 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1430/2829] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:07:42 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:07:42.707760 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f756d7-9c93-48d5-b1ad-41618881e88b admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:42.718506 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f756d7-9c93-48d5-b1ad-41618881e88b admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:42.723204 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb021855-991e-4ee8-b33f-1df3d77fbcd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:42.724114 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb021855-991e-4ee8-b33f-1df3d77fbcd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:42.724392 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb021855-991e-4ee8-b33f-1df3d77fbcd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:42.724664 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb021855-991e-4ee8-b33f-1df3d77fbcd9 admin admin] PATH_INFO: `/v3/projects/d50e1392921f459aada27b8319a1adb0/users/ee8f674890cb4f078ec87a889c5e4764/roles/7b20d65f49ee4a48894b6c060bb5d580` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:42.726685 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb021855-991e-4ee8-b33f-1df3d77fbcd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d50e1392921f459aada27b8319a1adb0, user_id=ee8f674890cb4f078ec87a889c5e4764, role_id=7b20d65f49ee4a48894b6c060bb5d580)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:42.729356 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb021855-991e-4ee8-b33f-1df3d77fbcd9 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:42.739614 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1400/2830] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 02:07:42 2022] PUT /identity/v3/projects/d50e1392921f459aada27b8319a1adb0/users/ee8f674890cb4f078ec87a889c5e4764/roles/7b20d65f49ee4a48894b6c060bb5d580 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:07:42.751057 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a298a6c5-9ef6-4716-9fa7-fade4d84e8fa None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:42.751369 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a298a6c5-9ef6-4716-9fa7-fade4d84e8fa None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:42.751646 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a298a6c5-9ef6-4716-9fa7-fade4d84e8fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:42.779580 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-a298a6c5-9ef6-4716-9fa7-fade4d84e8fa None None] MFA Rules not processed for user `ee8f674890cb4f078ec87a889c5e4764`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:07:42.780909 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-a298a6c5-9ef6-4716-9fa7-fade4d84e8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:07:42.785103 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:07:42.856392 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1431/2831] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:07:42 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:07:42.866996 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb021855-991e-4ee8-b33f-1df3d77fbcd9 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:42.869072 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-bb021855-991e-4ee8-b33f-1df3d77fbcd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:07:42.929323 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb021855-991e-4ee8-b33f-1df3d77fbcd9 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:42.933576 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3b95e6-21f4-4d44-8047-a323f148c541 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:42.934211 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b95e6-21f4-4d44-8047-a323f148c541 service cinder] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:42.936507 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b95e6-21f4-4d44-8047-a323f148c541 service cinder] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:42.936806 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b95e6-21f4-4d44-8047-a323f148c541 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:42.946508 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3b95e6-21f4-4d44-8047-a323f148c541 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:42.947973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3b95e6-21f4-4d44-8047-a323f148c541 service cinder] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:42.957467 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1401/2832] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:07:42 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:07:43.505440 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a298a6c5-9ef6-4716-9fa7-fade4d84e8fa None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:43.572183 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a298a6c5-9ef6-4716-9fa7-fade4d84e8fa None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:43.577215 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a846666-854d-4c74-ac62-baccca845a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:43.578079 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a846666-854d-4c74-ac62-baccca845a3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:43.578355 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a846666-854d-4c74-ac62-baccca845a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:43.578623 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a846666-854d-4c74-ac62-baccca845a3a admin admin] PATH_INFO: `/v3/users/60a8c17b37844db69bfa9d0befed526d` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:43.579858 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a846666-854d-4c74-ac62-baccca845a3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=60a8c17b37844db69bfa9d0befed526d)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:43.581826 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a846666-854d-4c74-ac62-baccca845a3a admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:43.632064 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-2a846666-854d-4c74-ac62-baccca845a3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60a8c17b37844db69bfa9d0befed526d'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:43.643954 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-2a846666-854d-4c74-ac62-baccca845a3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60a8c17b37844db69bfa9d0befed526d'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:43.652817 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-2a846666-854d-4c74-ac62-baccca845a3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60a8c17b37844db69bfa9d0befed526d'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:43.661005 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-2a846666-854d-4c74-ac62-baccca845a3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60a8c17b37844db69bfa9d0befed526d'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:43.661968 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-2a846666-854d-4c74-ac62-baccca845a3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60a8c17b37844db69bfa9d0befed526d'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:43.672028 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1432/2833] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 02:07:43 2022] DELETE /identity/v3/users/60a8c17b37844db69bfa9d0befed526d => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:07:43.676764 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb6f388-9c16-4659-991b-1e4d2d5f3221 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:43.677054 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb6f388-9c16-4659-991b-1e4d2d5f3221 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:43.677334 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb6f388-9c16-4659-991b-1e4d2d5f3221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:43.696265 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-cdb6f388-9c16-4659-991b-1e4d2d5f3221 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:07:43.697540 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-cdb6f388-9c16-4659-991b-1e4d2d5f3221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:07:43.723640 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:07:43.760580 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1402/2834] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:07:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:07:43.770405 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a846666-854d-4c74-ac62-baccca845a3a admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:43.772031 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-2a846666-854d-4c74-ac62-baccca845a3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:07:43.796114 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:07:43.832697 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a846666-854d-4c74-ac62-baccca845a3a admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:43.836754 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a04f05-07ba-4b75-9356-53b9a9b48fa8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:43.837392 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a04f05-07ba-4b75-9356-53b9a9b48fa8 service neutron] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:43.837673 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a04f05-07ba-4b75-9356-53b9a9b48fa8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:43.837943 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a04f05-07ba-4b75-9356-53b9a9b48fa8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:43.847118 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a04f05-07ba-4b75-9356-53b9a9b48fa8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:43.848589 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a04f05-07ba-4b75-9356-53b9a9b48fa8 service neutron] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:43.858564 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1433/2835] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:07:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:07:44.300738 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb6f388-9c16-4659-991b-1e4d2d5f3221 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:44.302453 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-cdb6f388-9c16-4659-991b-1e4d2d5f3221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:07:44.315088 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb6f388-9c16-4659-991b-1e4d2d5f3221 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:44.320256 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b350a41-cde2-4697-ac18-b27e64a55d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:44.321236 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b350a41-cde2-4697-ac18-b27e64a55d92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:44.321523 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b350a41-cde2-4697-ac18-b27e64a55d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:44.321795 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b350a41-cde2-4697-ac18-b27e64a55d92 admin admin] PATH_INFO: `/v3/projects/6045a976bcf3436dac4c596a10829ed7` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:44.323547 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b350a41-cde2-4697-ac18-b27e64a55d92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6045a976bcf3436dac4c596a10829ed7)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:44.325527 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b350a41-cde2-4697-ac18-b27e64a55d92 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:44.382434 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-4b350a41-cde2-4697-ac18-b27e64a55d92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6045a976bcf3436dac4c596a10829ed7', 'request_id': 'req-4b350a41-cde2-4697-ac18-b27e64a55d92'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:44.389085 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-4b350a41-cde2-4697-ac18-b27e64a55d92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6045a976bcf3436dac4c596a10829ed7', 'request_id': 'req-4b350a41-cde2-4697-ac18-b27e64a55d92'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:44.397895 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-4b350a41-cde2-4697-ac18-b27e64a55d92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6045a976bcf3436dac4c596a10829ed7', 'request_id': 'req-4b350a41-cde2-4697-ac18-b27e64a55d92'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:44.411078 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-4b350a41-cde2-4697-ac18-b27e64a55d92 admin admin] The token cache is being invalidate because project 6045a976bcf3436dac4c596a10829ed7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 02:07:44.411413 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-4b350a41-cde2-4697-ac18-b27e64a55d92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:44.412068 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-4b350a41-cde2-4697-ac18-b27e64a55d92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:44.413661 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1403/2836] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 02:07:44 2022] DELETE /identity/v3/projects/6045a976bcf3436dac4c596a10829ed7 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:07:44.417993 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a04f05-07ba-4b75-9356-53b9a9b48fa8 service neutron] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:44.429531 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a04f05-07ba-4b75-9356-53b9a9b48fa8 service neutron] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:44.490297 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e36de5-71bb-4446-8c1b-57e25e011e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:44.491336 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e36de5-71bb-4446-8c1b-57e25e011e79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:44.491679 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e36de5-71bb-4446-8c1b-57e25e011e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:44.492017 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e36de5-71bb-4446-8c1b-57e25e011e79 admin admin] PATH_INFO: `/v3/users/ee8f674890cb4f078ec87a889c5e4764` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:44.493298 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e36de5-71bb-4446-8c1b-57e25e011e79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee8f674890cb4f078ec87a889c5e4764)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:44.495340 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e36de5-71bb-4446-8c1b-57e25e011e79 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:44.554165 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-e2e36de5-71bb-4446-8c1b-57e25e011e79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee8f674890cb4f078ec87a889c5e4764'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:44.566892 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-e2e36de5-71bb-4446-8c1b-57e25e011e79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee8f674890cb4f078ec87a889c5e4764'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:44.576272 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-e2e36de5-71bb-4446-8c1b-57e25e011e79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee8f674890cb4f078ec87a889c5e4764'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:44.584529 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-e2e36de5-71bb-4446-8c1b-57e25e011e79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee8f674890cb4f078ec87a889c5e4764'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:44.585300 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-e2e36de5-71bb-4446-8c1b-57e25e011e79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee8f674890cb4f078ec87a889c5e4764'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:44.592084 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1434/2837] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 02:07:44 2022] DELETE /identity/v3/users/ee8f674890cb4f078ec87a889c5e4764 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:07:44.889357 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b350a41-cde2-4697-ac18-b27e64a55d92 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:44.890919 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-4b350a41-cde2-4697-ac18-b27e64a55d92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:07:44.904314 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b350a41-cde2-4697-ac18-b27e64a55d92 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:44.962625 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a9968d-c87b-492c-b126-ae349b89c8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:44.963568 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a9968d-c87b-492c-b126-ae349b89c8f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:44.963850 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a9968d-c87b-492c-b126-ae349b89c8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:44.964126 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a9968d-c87b-492c-b126-ae349b89c8f4 admin admin] PATH_INFO: `/v3/projects/d50e1392921f459aada27b8319a1adb0` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:44.965342 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a9968d-c87b-492c-b126-ae349b89c8f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d50e1392921f459aada27b8319a1adb0)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:44.967406 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a9968d-c87b-492c-b126-ae349b89c8f4 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:45.015902 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-f5a9968d-c87b-492c-b126-ae349b89c8f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd50e1392921f459aada27b8319a1adb0', 'request_id': 'req-f5a9968d-c87b-492c-b126-ae349b89c8f4'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:45.022002 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-f5a9968d-c87b-492c-b126-ae349b89c8f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd50e1392921f459aada27b8319a1adb0', 'request_id': 'req-f5a9968d-c87b-492c-b126-ae349b89c8f4'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:45.023854 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:07:45.032168 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-f5a9968d-c87b-492c-b126-ae349b89c8f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd50e1392921f459aada27b8319a1adb0', 'request_id': 'req-f5a9968d-c87b-492c-b126-ae349b89c8f4'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:45.046075 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-f5a9968d-c87b-492c-b126-ae349b89c8f4 admin admin] The token cache is being invalidate because project d50e1392921f459aada27b8319a1adb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 02:07:45.046400 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-f5a9968d-c87b-492c-b126-ae349b89c8f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:45.047150 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-f5a9968d-c87b-492c-b126-ae349b89c8f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:07:45.048828 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1404/2838] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 02:07:44 2022] DELETE /identity/v3/projects/d50e1392921f459aada27b8319a1adb0 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:07:45.053835 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d23bff-a641-457f-a638-7bf228e172d3 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:45.054310 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d23bff-a641-457f-a638-7bf228e172d3 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:45.054429 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d23bff-a641-457f-a638-7bf228e172d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:45.072711 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-98d23bff-a641-457f-a638-7bf228e172d3 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:07:45.073898 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-98d23bff-a641-457f-a638-7bf228e172d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:07:45.114424 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:07:45.139705 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1435/2839] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:07:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 02:07:45.148010 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d918284a-df64-484c-9f36-488bb9e35715 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:45.148303 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d918284a-df64-484c-9f36-488bb9e35715 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:45.148594 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d918284a-df64-484c-9f36-488bb9e35715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:45.168382 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-d918284a-df64-484c-9f36-488bb9e35715 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:07:45.169940 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-d918284a-df64-484c-9f36-488bb9e35715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:07:45.178477 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1405/2840] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:07:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:07:45.183061 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d23bff-a641-457f-a638-7bf228e172d3 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:45.193021 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d23bff-a641-457f-a638-7bf228e172d3 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:45.198133 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f334dd3f-ace6-4f53-971b-b81f72af1a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:45.198795 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f334dd3f-ace6-4f53-971b-b81f72af1a9e admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:45.199077 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f334dd3f-ace6-4f53-971b-b81f72af1a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:45.199385 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f334dd3f-ace6-4f53-971b-b81f72af1a9e admin admin] PATH_INFO: `/v3/domains` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:45.200146 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f334dd3f-ace6-4f53-971b-b81f72af1a9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80460) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 02:07:45.200454 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f334dd3f-ace6-4f53-971b-b81f72af1a9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:45.202083 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f334dd3f-ace6-4f53-971b-b81f72af1a9e admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:45.213826 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1436/2841] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 02:07:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 02:07:45.218244 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d918284a-df64-484c-9f36-488bb9e35715 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:45.228299 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d918284a-df64-484c-9f36-488bb9e35715 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:45.233250 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f69d9-cb40-4141-8b00-fd675b6989dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:45.233901 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f69d9-cb40-4141-8b00-fd675b6989dc admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:45.234184 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f69d9-cb40-4141-8b00-fd675b6989dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:45.234455 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f69d9-cb40-4141-8b00-fd675b6989dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:45.235149 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0f69d9-cb40-4141-8b00-fd675b6989dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:45.237138 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0f69d9-cb40-4141-8b00-fd675b6989dc admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:45.266674 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1406/2842] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 02:07:45 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:07:45.268108 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f334dd3f-ace6-4f53-971b-b81f72af1a9e admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:45.278486 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f334dd3f-ace6-4f53-971b-b81f72af1a9e admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:45.336879 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f5666-7230-4f3c-aed1-322b1dd23550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:45.337543 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f5666-7230-4f3c-aed1-322b1dd23550 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:45.337821 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f5666-7230-4f3c-aed1-322b1dd23550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:45.338095 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f5666-7230-4f3c-aed1-322b1dd23550 admin admin] PATH_INFO: `/v3/users` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:45.338792 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661f5666-7230-4f3c-aed1-322b1dd23550 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:45.340932 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661f5666-7230-4f3c-aed1-322b1dd23550 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:45.361346 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1437/2843] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 02:07:45 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:07:45.365499 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f69d9-cb40-4141-8b00-fd675b6989dc admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:45.376115 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f69d9-cb40-4141-8b00-fd675b6989dc admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:45.380938 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d116df-7588-4df4-87d6-42ed0611e7b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:45.381742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d116df-7588-4df4-87d6-42ed0611e7b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:45.382122 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d116df-7588-4df4-87d6-42ed0611e7b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:45.382464 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d116df-7588-4df4-87d6-42ed0611e7b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:45.383297 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d116df-7588-4df4-87d6-42ed0611e7b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:45.384973 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d116df-7588-4df4-87d6-42ed0611e7b8 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:45.397047 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1407/2844] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 02:07:45.401694 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f5666-7230-4f3c-aed1-322b1dd23550 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:45.412938 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f5666-7230-4f3c-aed1-322b1dd23550 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:45.418199 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb54531-da6e-49cc-a346-090574d1a420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:45.419334 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb54531-da6e-49cc-a346-090574d1a420 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:45.419622 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb54531-da6e-49cc-a346-090574d1a420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:45.419895 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb54531-da6e-49cc-a346-090574d1a420 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:45.420629 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb54531-da6e-49cc-a346-090574d1a420 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:45.422254 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb54531-da6e-49cc-a346-090574d1a420 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:45.433773 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1438/2845] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:07:45.437566 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d116df-7588-4df4-87d6-42ed0611e7b8 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:45.450350 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d116df-7588-4df4-87d6-42ed0611e7b8 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:45.455375 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2308c713-da77-4182-ac00-df92c24145b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:45.456269 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2308c713-da77-4182-ac00-df92c24145b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:45.456542 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2308c713-da77-4182-ac00-df92c24145b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:45.456810 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2308c713-da77-4182-ac00-df92c24145b7 admin admin] PATH_INFO: `/v3/projects/eb1c8afa40364b16ab0b2aa88f9472f9/users/0a90e5e4013c4a289bfb47f0ef0f97f2/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:45.472907 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2308c713-da77-4182-ac00-df92c24145b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb1c8afa40364b16ab0b2aa88f9472f9, user_id=0a90e5e4013c4a289bfb47f0ef0f97f2, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:45.475668 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2308c713-da77-4182-ac00-df92c24145b7 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:45.490027 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1408/2846] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 02:07:45 2022] PUT /identity/v3/projects/eb1c8afa40364b16ab0b2aa88f9472f9/users/0a90e5e4013c4a289bfb47f0ef0f97f2/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:07:45.495530 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa2eb7-f36c-4661-9f3d-b819ed06fcd3 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:45.495853 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa2eb7-f36c-4661-9f3d-b819ed06fcd3 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:45.496189 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa2eb7-f36c-4661-9f3d-b819ed06fcd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:45.514924 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-51fa2eb7-f36c-4661-9f3d-b819ed06fcd3 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:07:45.516166 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-51fa2eb7-f36c-4661-9f3d-b819ed06fcd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:07:45.578243 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1439/2847] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:07:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 02:07:45.589623 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2308c713-da77-4182-ac00-df92c24145b7 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:45.591418 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-2308c713-da77-4182-ac00-df92c24145b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:07:45.649367 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2308c713-da77-4182-ac00-df92c24145b7 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:45.652969 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a8770e-ae8d-47cb-85e3-0b3ff718cbce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:45.653630 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8770e-ae8d-47cb-85e3-0b3ff718cbce service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:45.653908 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8770e-ae8d-47cb-85e3-0b3ff718cbce service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:45.654177 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8770e-ae8d-47cb-85e3-0b3ff718cbce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:45.664210 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a8770e-ae8d-47cb-85e3-0b3ff718cbce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:45.665671 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a8770e-ae8d-47cb-85e3-0b3ff718cbce service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:45.676383 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1409/2848] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:07:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:07:46.288552 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee558265-a4fd-420a-ae7e-694590fea80a None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:46.289088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee558265-a4fd-420a-ae7e-694590fea80a None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:46.289088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee558265-a4fd-420a-ae7e-694590fea80a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:46.317880 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-ee558265-a4fd-420a-ae7e-694590fea80a None None] MFA Rules not processed for user `0a90e5e4013c4a289bfb47f0ef0f97f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:07:46.319070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-ee558265-a4fd-420a-ae7e-694590fea80a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:07:46.359487 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:07:46.386211 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1440/2849] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:07:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:07:46.397747 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a8770e-ae8d-47cb-85e3-0b3ff718cbce service neutron] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:46.399842 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-72a8770e-ae8d-47cb-85e3-0b3ff718cbce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:07:46.482175 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a8770e-ae8d-47cb-85e3-0b3ff718cbce service neutron] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:46.485746 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1509792-7cfc-4a4f-82ad-cbc484485f46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:46.486401 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1509792-7cfc-4a4f-82ad-cbc484485f46 service cinder] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:46.486678 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1509792-7cfc-4a4f-82ad-cbc484485f46 service cinder] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:46.486947 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1509792-7cfc-4a4f-82ad-cbc484485f46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:46.495046 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1509792-7cfc-4a4f-82ad-cbc484485f46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:46.496514 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1509792-7cfc-4a4f-82ad-cbc484485f46 service cinder] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:46.505160 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1410/2850] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:07:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:07:47.868751 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbb069-d0fc-43eb-9106-c730fa37f768 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:47.869246 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbb069-d0fc-43eb-9106-c730fa37f768 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:47.869346 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbb069-d0fc-43eb-9106-c730fa37f768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:47.888242 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-65bbb069-d0fc-43eb-9106-c730fa37f768 None None] MFA Rules not processed for user `0a90e5e4013c4a289bfb47f0ef0f97f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:07:47.889461 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-65bbb069-d0fc-43eb-9106-c730fa37f768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:07:47.897699 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1441/2851] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:07:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:07:48.396012 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1509792-7cfc-4a4f-82ad-cbc484485f46 service cinder] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:07:48.406544 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1509792-7cfc-4a4f-82ad-cbc484485f46 service cinder] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:07:48.410583 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051027af-26df-47b0-91c5-bdeaed4ee369 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:07:48.411374 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051027af-26df-47b0-91c5-bdeaed4ee369 service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:07:48.411656 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051027af-26df-47b0-91c5-bdeaed4ee369 service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:07:48.412008 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051027af-26df-47b0-91c5-bdeaed4ee369 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:07:48.414470 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-051027af-26df-47b0-91c5-bdeaed4ee369 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:07:48.425262 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051027af-26df-47b0-91c5-bdeaed4ee369 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:07:48.426833 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051027af-26df-47b0-91c5-bdeaed4ee369 service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:07:48.435177 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1411/2852] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:07:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:07:57.882458 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=13 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:07:58.431623 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=89 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:08:01.104100 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f5fb3c-eac9-4198-b4d0-f949f5e40066 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:08:01.104594 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f5fb3c-eac9-4198-b4d0-f949f5e40066 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:08:01.104594 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f5fb3c-eac9-4198-b4d0-f949f5e40066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:08:01.122354 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-02f5fb3c-eac9-4198-b4d0-f949f5e40066 None None] MFA Rules not processed for user `0a90e5e4013c4a289bfb47f0ef0f97f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:08:01.123574 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-02f5fb3c-eac9-4198-b4d0-f949f5e40066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:08:01.130695 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1442/2853] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:08:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:08:11.117073 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=3 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:08:17.266309 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d1e3a-f79b-40b0-ab9f-bf77c85f85da None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:08:17.266837 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d1e3a-f79b-40b0-ab9f-bf77c85f85da None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:08:17.266837 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d1e3a-f79b-40b0-ab9f-bf77c85f85da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:08:17.286896 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-de4d1e3a-f79b-40b0-ab9f-bf77c85f85da None None] MFA Rules not processed for user `168e6181ad22460990f1f7361b483364`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:08:17.288198 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-de4d1e3a-f79b-40b0-ab9f-bf77c85f85da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:08:17.352188 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1412/2854] 10.222.0.26 () {58 vars in 1009 bytes} [Thu Jun 23 02:08:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:08:17.401921 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f5fb3c-eac9-4198-b4d0-f949f5e40066 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:08:17.403575 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-02f5fb3c-eac9-4198-b4d0-f949f5e40066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:08:17.491933 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f5fb3c-eac9-4198-b4d0-f949f5e40066 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:08:17.497460 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641d7072-7d95-43e8-a9cb-a53fd2f3d50e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28e59688485241c4b6d51f99bbb3d059', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:08:17.498103 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d7072-7d95-43e8-a9cb-a53fd2f3d50e service swift] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:08:17.498377 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d7072-7d95-43e8-a9cb-a53fd2f3d50e service swift] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:08:17.498649 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d7072-7d95-43e8-a9cb-a53fd2f3d50e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:08:17.507030 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641d7072-7d95-43e8-a9cb-a53fd2f3d50e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:08:17.508667 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641d7072-7d95-43e8-a9cb-a53fd2f3d50e service swift] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:08:17.517226 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1443/2855] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Jun 23 02:08:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 23 02:08:27.340368 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=25 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:08:27.513501 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=32 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:08:32.185680 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c744083c-38cf-4792-ab97-d267ddda8763 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:08:32.186189 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c744083c-38cf-4792-ab97-d267ddda8763 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:08:32.186189 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c744083c-38cf-4792-ab97-d267ddda8763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:08:32.204700 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-c744083c-38cf-4792-ab97-d267ddda8763 None None] MFA Rules not processed for user `168e6181ad22460990f1f7361b483364`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:08:32.205927 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-c744083c-38cf-4792-ab97-d267ddda8763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:08:32.213499 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1413/2856] 10.222.0.26 () {58 vars in 967 bytes} [Thu Jun 23 02:08:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:08:32.225773 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641d7072-7d95-43e8-a9cb-a53fd2f3d50e service swift] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:08:32.236591 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641d7072-7d95-43e8-a9cb-a53fd2f3d50e service swift] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:08:32.243017 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a8fd0d-9208-42c8-8e62-2b6224a14f8a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '28e59688485241c4b6d51f99bbb3d059', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:08:32.243676 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8fd0d-9208-42c8-8e62-2b6224a14f8a service swift] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:08:32.243952 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8fd0d-9208-42c8-8e62-2b6224a14f8a service swift] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:08:32.244220 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8fd0d-9208-42c8-8e62-2b6224a14f8a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:08:32.253422 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a8fd0d-9208-42c8-8e62-2b6224a14f8a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:08:32.255008 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a8fd0d-9208-42c8-8e62-2b6224a14f8a service swift] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:08:32.263552 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1444/2857] 10.222.0.26 () {60 vars in 1384 bytes} [Thu Jun 23 02:08:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 02:08:34.210942 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c744083c-38cf-4792-ab97-d267ddda8763 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:08:34.224283 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c744083c-38cf-4792-ab97-d267ddda8763 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:08:34.228969 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa9347d-5f63-457d-bcb8-44ba77dafc58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:08:34.229732 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9347d-5f63-457d-bcb8-44ba77dafc58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:08:34.230009 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9347d-5f63-457d-bcb8-44ba77dafc58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:08:34.230278 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9347d-5f63-457d-bcb8-44ba77dafc58 admin admin] PATH_INFO: `/v3/users/0a90e5e4013c4a289bfb47f0ef0f97f2` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:08:34.231537 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa9347d-5f63-457d-bcb8-44ba77dafc58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a90e5e4013c4a289bfb47f0ef0f97f2)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:08:34.233490 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa9347d-5f63-457d-bcb8-44ba77dafc58 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:08:34.297119 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-6fa9347d-5f63-457d-bcb8-44ba77dafc58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a90e5e4013c4a289bfb47f0ef0f97f2'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:08:34.311201 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-6fa9347d-5f63-457d-bcb8-44ba77dafc58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a90e5e4013c4a289bfb47f0ef0f97f2'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:08:34.320049 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-6fa9347d-5f63-457d-bcb8-44ba77dafc58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a90e5e4013c4a289bfb47f0ef0f97f2'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:08:34.327200 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-6fa9347d-5f63-457d-bcb8-44ba77dafc58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a90e5e4013c4a289bfb47f0ef0f97f2'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:08:34.327903 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-6fa9347d-5f63-457d-bcb8-44ba77dafc58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a90e5e4013c4a289bfb47f0ef0f97f2'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:08:34.333990 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1414/2858] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 02:08:34 2022] DELETE /identity/v3/users/0a90e5e4013c4a289bfb47f0ef0f97f2 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:08:34.517925 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a8fd0d-9208-42c8-8e62-2b6224a14f8a service swift] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:08:34.529396 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a8fd0d-9208-42c8-8e62-2b6224a14f8a service swift] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:08:34.588243 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab8e906-c7ab-4e91-85fb-b5d00c78c57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:08:34.589105 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab8e906-c7ab-4e91-85fb-b5d00c78c57f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:08:34.589423 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab8e906-c7ab-4e91-85fb-b5d00c78c57f admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:08:34.589713 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab8e906-c7ab-4e91-85fb-b5d00c78c57f admin admin] PATH_INFO: `/v3/projects/eb1c8afa40364b16ab0b2aa88f9472f9` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:08:34.590953 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab8e906-c7ab-4e91-85fb-b5d00c78c57f admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb1c8afa40364b16ab0b2aa88f9472f9)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:08:34.592967 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab8e906-c7ab-4e91-85fb-b5d00c78c57f admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:08:34.641068 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-eab8e906-c7ab-4e91-85fb-b5d00c78c57f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb1c8afa40364b16ab0b2aa88f9472f9', 'request_id': 'req-eab8e906-c7ab-4e91-85fb-b5d00c78c57f'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:08:34.649151 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-eab8e906-c7ab-4e91-85fb-b5d00c78c57f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb1c8afa40364b16ab0b2aa88f9472f9', 'request_id': 'req-eab8e906-c7ab-4e91-85fb-b5d00c78c57f'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:08:34.658792 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-eab8e906-c7ab-4e91-85fb-b5d00c78c57f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb1c8afa40364b16ab0b2aa88f9472f9', 'request_id': 'req-eab8e906-c7ab-4e91-85fb-b5d00c78c57f'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:08:34.670874 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-eab8e906-c7ab-4e91-85fb-b5d00c78c57f admin admin] The token cache is being invalidate because project eb1c8afa40364b16ab0b2aa88f9472f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 02:08:34.671277 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-eab8e906-c7ab-4e91-85fb-b5d00c78c57f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:08:34.672175 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-eab8e906-c7ab-4e91-85fb-b5d00c78c57f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:08:34.674591 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1445/2859] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 02:08:34 2022] DELETE /identity/v3/projects/eb1c8afa40364b16ab0b2aa88f9472f9 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:08:34.680535 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690f381-d337-461f-a2c8-4b06c18dfc29 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:08:34.681036 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690f381-d337-461f-a2c8-4b06c18dfc29 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:08:34.681398 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690f381-d337-461f-a2c8-4b06c18dfc29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:08:34.700999 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-d690f381-d337-461f-a2c8-4b06c18dfc29 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:08:34.702251 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-d690f381-d337-461f-a2c8-4b06c18dfc29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:08:34.769777 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1415/2860] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:08:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:08:34.778286 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae3aad-9e04-466a-870e-eef4124fa0d2 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:08:34.778628 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae3aad-9e04-466a-870e-eef4124fa0d2 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:08:34.778905 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae3aad-9e04-466a-870e-eef4124fa0d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:08:34.798127 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-21ae3aad-9e04-466a-870e-eef4124fa0d2 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:08:34.799393 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-21ae3aad-9e04-466a-870e-eef4124fa0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:08:34.808514 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1446/2861] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:08:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:08:34.813633 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d690f381-d337-461f-a2c8-4b06c18dfc29 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:08:34.823712 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d690f381-d337-461f-a2c8-4b06c18dfc29 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:08:34.828740 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005cc825-0d0d-403a-a416-f1c2ef341a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:08:34.829388 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005cc825-0d0d-403a-a416-f1c2ef341a3a admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:08:34.829671 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005cc825-0d0d-403a-a416-f1c2ef341a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:08:34.829948 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005cc825-0d0d-403a-a416-f1c2ef341a3a admin admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:08:34.830989 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005cc825-0d0d-403a-a416-f1c2ef341a3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 02:08:34.830989 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005cc825-0d0d-403a-a416-f1c2ef341a3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:08:34.832644 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005cc825-0d0d-403a-a416-f1c2ef341a3a admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:08:34.844606 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1416/2862] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 02:08:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 02:08:34.849211 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ae3aad-9e04-466a-870e-eef4124fa0d2 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:08:34.859261 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ae3aad-9e04-466a-870e-eef4124fa0d2 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:08:34.864926 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6215f95-913e-4d02-a8b2-e05e626f57f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:08:34.865578 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6215f95-913e-4d02-a8b2-e05e626f57f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:08:34.865855 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6215f95-913e-4d02-a8b2-e05e626f57f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:08:34.866125 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6215f95-913e-4d02-a8b2-e05e626f57f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:08:34.866833 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6215f95-913e-4d02-a8b2-e05e626f57f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:08:34.868886 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6215f95-913e-4d02-a8b2-e05e626f57f7 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:08:34.896945 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1447/2863] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 02:08:34 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:08:34.900897 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005cc825-0d0d-403a-a416-f1c2ef341a3a admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:08:34.910873 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005cc825-0d0d-403a-a416-f1c2ef341a3a admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:08:34.959880 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:08:34.970391 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66215f3b-0fbd-49d8-ae1a-fa91c30da90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:08:34.971249 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66215f3b-0fbd-49d8-ae1a-fa91c30da90a admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:08:34.971717 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66215f3b-0fbd-49d8-ae1a-fa91c30da90a admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:08:34.972050 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66215f3b-0fbd-49d8-ae1a-fa91c30da90a admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:08:34.973585 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66215f3b-0fbd-49d8-ae1a-fa91c30da90a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:08:34.976725 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66215f3b-0fbd-49d8-ae1a-fa91c30da90a admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:08:35.000485 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1417/2864] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 02:08:34 2022] POST /identity/v3/users => generated 286 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:08:35.004779 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6215f95-913e-4d02-a8b2-e05e626f57f7 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:08:35.015699 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6215f95-913e-4d02-a8b2-e05e626f57f7 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:08:35.020617 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6e2eea-f14a-4a18-ae89-88531b42f5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:08:35.021202 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e2eea-f14a-4a18-ae89-88531b42f5e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:08:35.021478 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e2eea-f14a-4a18-ae89-88531b42f5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:08:35.021745 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e2eea-f14a-4a18-ae89-88531b42f5e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:08:35.022467 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6e2eea-f14a-4a18-ae89-88531b42f5e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:08:35.024080 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6e2eea-f14a-4a18-ae89-88531b42f5e2 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:08:35.034541 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1448/2865] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:08:35.038473 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66215f3b-0fbd-49d8-ae1a-fa91c30da90a admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:08:35.049017 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66215f3b-0fbd-49d8-ae1a-fa91c30da90a admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:08:35.053899 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd1cd0d-19d7-47f7-9b34-db5355258f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:08:35.054546 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd1cd0d-19d7-47f7-9b34-db5355258f57 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:08:35.054830 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd1cd0d-19d7-47f7-9b34-db5355258f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:08:35.055093 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd1cd0d-19d7-47f7-9b34-db5355258f57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:08:35.055864 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd1cd0d-19d7-47f7-9b34-db5355258f57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:08:35.057445 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd1cd0d-19d7-47f7-9b34-db5355258f57 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:08:35.067562 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1418/2866] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:08:35.071599 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6e2eea-f14a-4a18-ae89-88531b42f5e2 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:08:35.081753 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6e2eea-f14a-4a18-ae89-88531b42f5e2 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:08:35.087170 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617bfee1-33e8-490a-8bb6-00c805318fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:08:35.088067 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617bfee1-33e8-490a-8bb6-00c805318fa2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:08:35.088354 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617bfee1-33e8-490a-8bb6-00c805318fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:08:35.088623 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617bfee1-33e8-490a-8bb6-00c805318fa2 admin admin] PATH_INFO: `/v3/projects/0574eef26e4240a58956006c43149c00/users/4ab0cdb0cd8643fa9640af686129ee81/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:08:35.103141 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617bfee1-33e8-490a-8bb6-00c805318fa2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0574eef26e4240a58956006c43149c00, user_id=4ab0cdb0cd8643fa9640af686129ee81, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:08:35.105824 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617bfee1-33e8-490a-8bb6-00c805318fa2 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:08:35.120417 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1449/2867] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 02:08:35 2022] PUT /identity/v3/projects/0574eef26e4240a58956006c43149c00/users/4ab0cdb0cd8643fa9640af686129ee81/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:08:35.130455 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104a764-5572-44dd-b860-5642bd2b52b5 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:08:35.130742 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104a764-5572-44dd-b860-5642bd2b52b5 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:08:35.131022 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104a764-5572-44dd-b860-5642bd2b52b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:08:35.159051 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-f104a764-5572-44dd-b860-5642bd2b52b5 None None] MFA Rules not processed for user `4ab0cdb0cd8643fa9640af686129ee81`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:08:35.160299 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-f104a764-5572-44dd-b860-5642bd2b52b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:08:35.231721 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1419/2868] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:08:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:08:35.248229 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617bfee1-33e8-490a-8bb6-00c805318fa2 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:08:35.250270 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-617bfee1-33e8-490a-8bb6-00c805318fa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:08:35.266115 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:08:35.310422 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617bfee1-33e8-490a-8bb6-00c805318fa2 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:08:35.314255 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c2fe43-2b7a-47e1-954b-daf578d1d6e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:08:35.314896 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c2fe43-2b7a-47e1-954b-daf578d1d6e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:08:35.315176 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c2fe43-2b7a-47e1-954b-daf578d1d6e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:08:35.315493 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c2fe43-2b7a-47e1-954b-daf578d1d6e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:08:35.323974 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c2fe43-2b7a-47e1-954b-daf578d1d6e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:08:35.325443 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c2fe43-2b7a-47e1-954b-daf578d1d6e4 service cinder] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:08:35.334314 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1450/2869] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:08:35 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:08:35.358738 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f104a764-5572-44dd-b860-5642bd2b52b5 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:08:35.423639 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f104a764-5572-44dd-b860-5642bd2b52b5 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:08:35.429217 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1396eed-3642-4610-aced-c4813d1cb345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:08:35.430085 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1396eed-3642-4610-aced-c4813d1cb345 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:08:35.430365 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1396eed-3642-4610-aced-c4813d1cb345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:08:35.430642 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1396eed-3642-4610-aced-c4813d1cb345 admin admin] PATH_INFO: `/v3/users/4ab0cdb0cd8643fa9640af686129ee81` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:08:35.432003 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1396eed-3642-4610-aced-c4813d1cb345 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ab0cdb0cd8643fa9640af686129ee81)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:08:35.433927 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1396eed-3642-4610-aced-c4813d1cb345 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:08:35.489303 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-b1396eed-3642-4610-aced-c4813d1cb345 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ab0cdb0cd8643fa9640af686129ee81'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:08:35.500764 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:08:35.502294 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-b1396eed-3642-4610-aced-c4813d1cb345 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ab0cdb0cd8643fa9640af686129ee81'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:08:35.511792 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-b1396eed-3642-4610-aced-c4813d1cb345 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ab0cdb0cd8643fa9640af686129ee81'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:08:35.518856 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-b1396eed-3642-4610-aced-c4813d1cb345 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ab0cdb0cd8643fa9640af686129ee81'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:08:35.519519 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-b1396eed-3642-4610-aced-c4813d1cb345 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ab0cdb0cd8643fa9640af686129ee81'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:08:35.525528 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1420/2870] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 02:08:35 2022] DELETE /identity/v3/users/4ab0cdb0cd8643fa9640af686129ee81 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:08:35.530215 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d0b06-841a-41f7-9df9-c82315682ef7 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:08:35.530743 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d0b06-841a-41f7-9df9-c82315682ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:08:35.530743 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81d0b06-841a-41f7-9df9-c82315682ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:08:35.549123 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-a81d0b06-841a-41f7-9df9-c82315682ef7 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:08:35.550264 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-a81d0b06-841a-41f7-9df9-c82315682ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:08:35.615237 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1451/2871] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:08:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:08:35.625293 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1396eed-3642-4610-aced-c4813d1cb345 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:08:35.626891 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-b1396eed-3642-4610-aced-c4813d1cb345 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:08:35.699745 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1396eed-3642-4610-aced-c4813d1cb345 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:08:35.703582 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19410bb4-fe73-44b2-b3a8-3a425aa6035a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:08:35.704229 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19410bb4-fe73-44b2-b3a8-3a425aa6035a service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:08:35.704506 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19410bb4-fe73-44b2-b3a8-3a425aa6035a service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:08:35.704784 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19410bb4-fe73-44b2-b3a8-3a425aa6035a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:08:35.714312 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19410bb4-fe73-44b2-b3a8-3a425aa6035a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:08:35.715752 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19410bb4-fe73-44b2-b3a8-3a425aa6035a service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:08:35.729409 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1421/2872] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:08:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:08:36.004784 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d0b06-841a-41f7-9df9-c82315682ef7 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:08:36.006379 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-a81d0b06-841a-41f7-9df9-c82315682ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:08:36.019078 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81d0b06-841a-41f7-9df9-c82315682ef7 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:08:36.025201 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8426c331-c840-4812-9be1-41b71aa3dde2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:08:36.026093 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8426c331-c840-4812-9be1-41b71aa3dde2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:08:36.026369 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8426c331-c840-4812-9be1-41b71aa3dde2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:08:36.026638 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8426c331-c840-4812-9be1-41b71aa3dde2 admin admin] PATH_INFO: `/v3/projects/0574eef26e4240a58956006c43149c00` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:08:36.027914 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8426c331-c840-4812-9be1-41b71aa3dde2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0574eef26e4240a58956006c43149c00)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:08:36.029901 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8426c331-c840-4812-9be1-41b71aa3dde2 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:08:36.080568 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-8426c331-c840-4812-9be1-41b71aa3dde2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0574eef26e4240a58956006c43149c00', 'request_id': 'req-8426c331-c840-4812-9be1-41b71aa3dde2'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:08:36.087613 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-8426c331-c840-4812-9be1-41b71aa3dde2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0574eef26e4240a58956006c43149c00', 'request_id': 'req-8426c331-c840-4812-9be1-41b71aa3dde2'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:08:36.096380 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-8426c331-c840-4812-9be1-41b71aa3dde2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0574eef26e4240a58956006c43149c00', 'request_id': 'req-8426c331-c840-4812-9be1-41b71aa3dde2'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:08:36.107127 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-8426c331-c840-4812-9be1-41b71aa3dde2 admin admin] The token cache is being invalidate because project 0574eef26e4240a58956006c43149c00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 02:08:36.107422 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-8426c331-c840-4812-9be1-41b71aa3dde2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:08:36.108075 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-8426c331-c840-4812-9be1-41b71aa3dde2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:08:36.109630 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1452/2873] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 02:08:36 2022] DELETE /identity/v3/projects/0574eef26e4240a58956006c43149c00 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:08:36.114673 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc39bb-c9b2-4533-80df-a1c655420e8d None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:08:36.115008 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc39bb-c9b2-4533-80df-a1c655420e8d None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:08:36.115329 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc39bb-c9b2-4533-80df-a1c655420e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:08:36.132888 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-89fc39bb-c9b2-4533-80df-a1c655420e8d None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:08:36.134128 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-89fc39bb-c9b2-4533-80df-a1c655420e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:08:36.195471 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1422/2874] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:08:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:08:36.203196 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eacf1b-79a6-4e51-86a4-3d703cc8cc44 None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:08:36.203558 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eacf1b-79a6-4e51-86a4-3d703cc8cc44 None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:08:36.203838 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eacf1b-79a6-4e51-86a4-3d703cc8cc44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:08:36.222903 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-19eacf1b-79a6-4e51-86a4-3d703cc8cc44 None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:08:36.224153 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-19eacf1b-79a6-4e51-86a4-3d703cc8cc44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:08:36.233970 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1453/2875] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:08:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:08:36.239066 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fc39bb-c9b2-4533-80df-a1c655420e8d None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:08:36.252111 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fc39bb-c9b2-4533-80df-a1c655420e8d None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:08:36.257911 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf5b814-8558-4331-a2fe-ddc8d15e9ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:08:36.258568 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5b814-8558-4331-a2fe-ddc8d15e9ce3 admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:08:36.258852 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5b814-8558-4331-a2fe-ddc8d15e9ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:08:36.259120 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5b814-8558-4331-a2fe-ddc8d15e9ce3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:08:36.259885 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf5b814-8558-4331-a2fe-ddc8d15e9ce3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 02:08:36.260190 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf5b814-8558-4331-a2fe-ddc8d15e9ce3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:08:36.261843 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf5b814-8558-4331-a2fe-ddc8d15e9ce3 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:08:36.274325 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1423/2876] 10.222.0.26 () {58 vars in 1152 bytes} [Thu Jun 23 02:08:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 02:08:36.278967 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eacf1b-79a6-4e51-86a4-3d703cc8cc44 None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:08:36.288841 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eacf1b-79a6-4e51-86a4-3d703cc8cc44 None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:08:36.294343 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb6672-4c5f-481f-b7c1-84854f4995b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:08:36.295399 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb6672-4c5f-481f-b7c1-84854f4995b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:08:36.295675 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb6672-4c5f-481f-b7c1-84854f4995b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:08:36.295931 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb6672-4c5f-481f-b7c1-84854f4995b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:08:36.296737 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cb6672-4c5f-481f-b7c1-84854f4995b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:08:36.298631 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cb6672-4c5f-481f-b7c1-84854f4995b8 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:08:36.326944 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1454/2877] 10.222.0.26 () {60 vars in 1152 bytes} [Thu Jun 23 02:08:36 2022] POST /identity/v3/projects => generated 389 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 02:08:36.331420 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf5b814-8558-4331-a2fe-ddc8d15e9ce3 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:08:36.341856 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf5b814-8558-4331-a2fe-ddc8d15e9ce3 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:08:36.399589 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d5c0c-6762-40af-85e1-b2d53eb4a1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:08:36.400251 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d5c0c-6762-40af-85e1-b2d53eb4a1ad admin admin] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:08:36.400566 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d5c0c-6762-40af-85e1-b2d53eb4a1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:08:36.400838 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d5c0c-6762-40af-85e1-b2d53eb4a1ad admin admin] PATH_INFO: `/v3/users` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:08:36.401531 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d5c0c-6762-40af-85e1-b2d53eb4a1ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:08:36.403525 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d5c0c-6762-40af-85e1-b2d53eb4a1ad admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:08:36.419326 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:08:36.429203 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1424/2878] 10.222.0.26 () {60 vars in 1143 bytes} [Thu Jun 23 02:08:36 2022] POST /identity/v3/users => generated 295 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 02:08:36.430499 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb6672-4c5f-481f-b7c1-84854f4995b8 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:08:36.441580 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb6672-4c5f-481f-b7c1-84854f4995b8 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:08:36.447193 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbca2fb-e960-4026-b069-edfc27e900cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:08:36.447888 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbca2fb-e960-4026-b069-edfc27e900cc admin admin] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:08:36.448169 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbca2fb-e960-4026-b069-edfc27e900cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:08:36.448441 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbca2fb-e960-4026-b069-edfc27e900cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:08:36.449161 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbca2fb-e960-4026-b069-edfc27e900cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:08:36.450819 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbca2fb-e960-4026-b069-edfc27e900cc admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:08:36.456363 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:08:36.464070 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1455/2879] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:08:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:08:36.468468 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d5c0c-6762-40af-85e1-b2d53eb4a1ad admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:08:36.481054 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d5c0c-6762-40af-85e1-b2d53eb4a1ad admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:08:36.486359 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac903c2-097c-40c6-bf4d-ee89161ec43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:08:36.487034 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac903c2-097c-40c6-bf4d-ee89161ec43d admin admin] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:08:36.487353 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac903c2-097c-40c6-bf4d-ee89161ec43d admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:08:36.487631 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac903c2-097c-40c6-bf4d-ee89161ec43d admin admin] PATH_INFO: `/v3/roles` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:08:36.488370 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac903c2-097c-40c6-bf4d-ee89161ec43d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:08:36.489962 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac903c2-097c-40c6-bf4d-ee89161ec43d admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:08:36.500466 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1425/2880] 10.222.0.26 () {58 vars in 1121 bytes} [Thu Jun 23 02:08:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 02:08:36.504830 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbca2fb-e960-4026-b069-edfc27e900cc admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:08:36.514846 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbca2fb-e960-4026-b069-edfc27e900cc admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:08:36.519762 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f47fbe-a04f-40bf-874f-fe133d92db7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:08:36.520631 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f47fbe-a04f-40bf-874f-fe133d92db7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:08:36.520912 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f47fbe-a04f-40bf-874f-fe133d92db7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:08:36.521182 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f47fbe-a04f-40bf-874f-fe133d92db7e admin admin] PATH_INFO: `/v3/projects/61c669cabfd542e5942bd7a24b79e2cd/users/afeb71b7fc74482aafe5a0c72c589ad9/roles/9021e3c4e3384a309951be4cc1d852e5` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:08:36.536296 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f47fbe-a04f-40bf-874f-fe133d92db7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=61c669cabfd542e5942bd7a24b79e2cd, user_id=afeb71b7fc74482aafe5a0c72c589ad9, role_id=9021e3c4e3384a309951be4cc1d852e5)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:08:36.538965 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f47fbe-a04f-40bf-874f-fe133d92db7e admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:08:36.553808 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1456/2881] 10.222.0.26 () {60 vars in 1482 bytes} [Thu Jun 23 02:08:36 2022] PUT /identity/v3/projects/61c669cabfd542e5942bd7a24b79e2cd/users/afeb71b7fc74482aafe5a0c72c589ad9/roles/9021e3c4e3384a309951be4cc1d852e5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:08:36.615867 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e080e1-77e3-49c2-9e89-7563b4f29871 None None] REQUEST_METHOD: `POST` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:08:36.616122 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e080e1-77e3-49c2-9e89-7563b4f29871 None None] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:08:36.616406 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e080e1-77e3-49c2-9e89-7563b4f29871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:08:36.647823 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.auth.core [None req-53e080e1-77e3-49c2-9e89-7563b4f29871 None None] MFA Rules not processed for user `afeb71b7fc74482aafe5a0c72c589ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:08:36.649046 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-53e080e1-77e3-49c2-9e89-7563b4f29871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:08:36.714882 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1426/2882] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:08:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:08:36.726090 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f47fbe-a04f-40bf-874f-fe133d92db7e admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:08:36.727920 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-c1f47fbe-a04f-40bf-874f-fe133d92db7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:08:36.792921 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f47fbe-a04f-40bf-874f-fe133d92db7e admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:08:36.797602 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d52ef5-4ee4-4d14-856c-624b34ecfcb2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc5bb02f8eb64ac980e3218470b3c1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:08:36.798245 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d52ef5-4ee4-4d14-856c-624b34ecfcb2 service cinder] REQUEST_METHOD: `GET` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:08:36.798521 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d52ef5-4ee4-4d14-856c-624b34ecfcb2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:08:36.798793 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d52ef5-4ee4-4d14-856c-624b34ecfcb2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:08:36.808432 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d52ef5-4ee4-4d14-856c-624b34ecfcb2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:08:36.809996 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d52ef5-4ee4-4d14-856c-624b34ecfcb2 service cinder] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:08:36.819266 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1457/2883] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:08:36 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:08:46.711926 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=37 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:08:46.815648 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=36,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:08:53.171721 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e080e1-77e3-49c2-9e89-7563b4f29871 None None] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:08:53.239667 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e080e1-77e3-49c2-9e89-7563b4f29871 None None] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:08:53.245248 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d213a91b-2d50-4b25-8e6b-b6e77ad935be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:08:53.246127 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213a91b-2d50-4b25-8e6b-b6e77ad935be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:08:53.246415 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213a91b-2d50-4b25-8e6b-b6e77ad935be admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:08:53.246691 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213a91b-2d50-4b25-8e6b-b6e77ad935be admin admin] PATH_INFO: `/v3/users/afeb71b7fc74482aafe5a0c72c589ad9` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:08:53.247991 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d213a91b-2d50-4b25-8e6b-b6e77ad935be admin admin] RBAC: Authorizing `identity:delete_user(user_id=afeb71b7fc74482aafe5a0c72c589ad9)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:08:53.249936 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d213a91b-2d50-4b25-8e6b-b6e77ad935be admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:08:53.312010 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-d213a91b-2d50-4b25-8e6b-b6e77ad935be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afeb71b7fc74482aafe5a0c72c589ad9'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:08:53.326673 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-d213a91b-2d50-4b25-8e6b-b6e77ad935be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afeb71b7fc74482aafe5a0c72c589ad9'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:08:53.335971 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-d213a91b-2d50-4b25-8e6b-b6e77ad935be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afeb71b7fc74482aafe5a0c72c589ad9'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:08:53.343197 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-d213a91b-2d50-4b25-8e6b-b6e77ad935be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afeb71b7fc74482aafe5a0c72c589ad9'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:08:53.343923 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-d213a91b-2d50-4b25-8e6b-b6e77ad935be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afeb71b7fc74482aafe5a0c72c589ad9'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:08:53.350461 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1427/2884] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 02:08:53 2022] DELETE /identity/v3/users/afeb71b7fc74482aafe5a0c72c589ad9 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:08:53.355158 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e17da-facf-40c6-ac56-765d31da94fc None None] REQUEST_METHOD: `POST` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:08:53.355694 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e17da-facf-40c6-ac56-765d31da94fc None None] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:08:53.355795 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e17da-facf-40c6-ac56-765d31da94fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:08:53.373864 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.auth.core [None req-633e17da-facf-40c6-ac56-765d31da94fc None None] MFA Rules not processed for user `34e5aabcf8f5425f9c9f312f635ff77a`. Rule list: `[]` (Enabled: `True`). {{(pid=80460) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 02:08:53.375088 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-633e17da-facf-40c6-ac56-765d31da94fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:08:53.436759 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1458/2885] 10.222.0.26 () {58 vars in 957 bytes} [Thu Jun 23 02:08:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 02:08:53.446936 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d213a91b-2d50-4b25-8e6b-b6e77ad935be admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:08:53.448625 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-d213a91b-2d50-4b25-8e6b-b6e77ad935be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:08:53.509438 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d213a91b-2d50-4b25-8e6b-b6e77ad935be admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:08:53.514032 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f5a0d4-a346-4227-82cd-cdecc0eda679 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '948e4905de44473bb7a45e1b521cf095', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:08:53.514686 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5a0d4-a346-4227-82cd-cdecc0eda679 service neutron] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:08:53.514971 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5a0d4-a346-4227-82cd-cdecc0eda679 service neutron] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:08:53.515278 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5a0d4-a346-4227-82cd-cdecc0eda679 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:08:53.526006 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f5a0d4-a346-4227-82cd-cdecc0eda679 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:08:53.528225 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f5a0d4-a346-4227-82cd-cdecc0eda679 service neutron] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:08:53.543201 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1428/2886] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:08:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:08:53.816898 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633e17da-facf-40c6-ac56-765d31da94fc None None] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:08:53.818360 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-633e17da-facf-40c6-ac56-765d31da94fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:08:53.833605 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633e17da-facf-40c6-ac56-765d31da94fc None None] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:08:53.838916 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e289f707-e473-46a7-ac1b-3cd6d097bd22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:08:53.839805 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e289f707-e473-46a7-ac1b-3cd6d097bd22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:08:53.840098 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e289f707-e473-46a7-ac1b-3cd6d097bd22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:08:53.840370 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e289f707-e473-46a7-ac1b-3cd6d097bd22 admin admin] PATH_INFO: `/v3/projects/61c669cabfd542e5942bd7a24b79e2cd` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:08:53.841812 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e289f707-e473-46a7-ac1b-3cd6d097bd22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61c669cabfd542e5942bd7a24b79e2cd)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:08:53.843731 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e289f707-e473-46a7-ac1b-3cd6d097bd22 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:08:53.892071 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-e289f707-e473-46a7-ac1b-3cd6d097bd22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61c669cabfd542e5942bd7a24b79e2cd', 'request_id': 'req-e289f707-e473-46a7-ac1b-3cd6d097bd22'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:08:53.898251 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-e289f707-e473-46a7-ac1b-3cd6d097bd22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61c669cabfd542e5942bd7a24b79e2cd', 'request_id': 'req-e289f707-e473-46a7-ac1b-3cd6d097bd22'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:08:53.909177 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-e289f707-e473-46a7-ac1b-3cd6d097bd22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61c669cabfd542e5942bd7a24b79e2cd', 'request_id': 'req-e289f707-e473-46a7-ac1b-3cd6d097bd22'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:08:53.920638 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-e289f707-e473-46a7-ac1b-3cd6d097bd22 admin admin] The token cache is being invalidate because project 61c669cabfd542e5942bd7a24b79e2cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 02:08:53.920925 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-e289f707-e473-46a7-ac1b-3cd6d097bd22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:08:53.921565 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-e289f707-e473-46a7-ac1b-3cd6d097bd22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:08:53.923140 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1459/2887] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 02:08:53 2022] DELETE /identity/v3/projects/61c669cabfd542e5942bd7a24b79e2cd => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:09:03.535962 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:09:03.917317 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:09:13.953623 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f5a0d4-a346-4227-82cd-cdecc0eda679 service neutron] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:09:13.955061 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-a7f5a0d4-a346-4227-82cd-cdecc0eda679 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:09:14.047836 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f5a0d4-a346-4227-82cd-cdecc0eda679 service neutron] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:09:14.106173 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7946c28-7eb5-4947-bc9c-253974312a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:09:14.107062 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7946c28-7eb5-4947-bc9c-253974312a16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:09:14.107368 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7946c28-7eb5-4947-bc9c-253974312a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:09:14.107642 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7946c28-7eb5-4947-bc9c-253974312a16 admin admin] PATH_INFO: `/v3/users/2b55b66ac25848bda892d86d4d0d6f4e` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:09:14.108846 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7946c28-7eb5-4947-bc9c-253974312a16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b55b66ac25848bda892d86d4d0d6f4e)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:09:14.110789 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7946c28-7eb5-4947-bc9c-253974312a16 admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:09:14.162591 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-d7946c28-7eb5-4947-bc9c-253974312a16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b55b66ac25848bda892d86d4d0d6f4e'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:09:14.179481 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-d7946c28-7eb5-4947-bc9c-253974312a16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b55b66ac25848bda892d86d4d0d6f4e'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:09:14.188765 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-d7946c28-7eb5-4947-bc9c-253974312a16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b55b66ac25848bda892d86d4d0d6f4e'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:09:14.196061 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-d7946c28-7eb5-4947-bc9c-253974312a16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b55b66ac25848bda892d86d4d0d6f4e'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:09:14.196669 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-d7946c28-7eb5-4947-bc9c-253974312a16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b55b66ac25848bda892d86d4d0d6f4e'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:09:14.202651 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1429/2888] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 02:09:13 2022] DELETE /identity/v3/users/2b55b66ac25848bda892d86d4d0d6f4e => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:09:14.388021 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e289f707-e473-46a7-ac1b-3cd6d097bd22 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:09:14.389538 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-e289f707-e473-46a7-ac1b-3cd6d097bd22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:09:14.450114 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e289f707-e473-46a7-ac1b-3cd6d097bd22 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:09:14.503992 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d45f23-bf3e-4cdc-bc0f-cd2c206e30b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:09:14.504841 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d45f23-bf3e-4cdc-bc0f-cd2c206e30b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:09:14.505118 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d45f23-bf3e-4cdc-bc0f-cd2c206e30b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:09:14.505389 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d45f23-bf3e-4cdc-bc0f-cd2c206e30b4 admin admin] PATH_INFO: `/v3/projects/136fd75d8a874bb3aa24ef1564f8e36b` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:09:14.506655 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d45f23-bf3e-4cdc-bc0f-cd2c206e30b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=136fd75d8a874bb3aa24ef1564f8e36b)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:09:14.508593 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d45f23-bf3e-4cdc-bc0f-cd2c206e30b4 admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:09:14.557366 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-72d45f23-bf3e-4cdc-bc0f-cd2c206e30b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '136fd75d8a874bb3aa24ef1564f8e36b', 'request_id': 'req-72d45f23-bf3e-4cdc-bc0f-cd2c206e30b4'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:09:14.563549 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-72d45f23-bf3e-4cdc-bc0f-cd2c206e30b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '136fd75d8a874bb3aa24ef1564f8e36b', 'request_id': 'req-72d45f23-bf3e-4cdc-bc0f-cd2c206e30b4'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:09:14.573282 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-72d45f23-bf3e-4cdc-bc0f-cd2c206e30b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '136fd75d8a874bb3aa24ef1564f8e36b', 'request_id': 'req-72d45f23-bf3e-4cdc-bc0f-cd2c206e30b4'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:09:14.583776 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-72d45f23-bf3e-4cdc-bc0f-cd2c206e30b4 admin admin] The token cache is being invalidate because project 136fd75d8a874bb3aa24ef1564f8e36b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 02:09:14.584047 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-72d45f23-bf3e-4cdc-bc0f-cd2c206e30b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:09:14.584716 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-72d45f23-bf3e-4cdc-bc0f-cd2c206e30b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:09:14.586504 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1460/2889] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 02:09:14 2022] DELETE /identity/v3/projects/136fd75d8a874bb3aa24ef1564f8e36b => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:09:14.590814 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7946c28-7eb5-4947-bc9c-253974312a16 admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:09:14.592610 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-d7946c28-7eb5-4947-bc9c-253974312a16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:09:14.605959 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7946c28-7eb5-4947-bc9c-253974312a16 admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:09:14.618875 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:09:14.664566 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77305b4-b94d-4c2b-ae2d-c20c7f76d6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:09:14.665502 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77305b4-b94d-4c2b-ae2d-c20c7f76d6bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:09:14.665810 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77305b4-b94d-4c2b-ae2d-c20c7f76d6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:09:14.666085 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77305b4-b94d-4c2b-ae2d-c20c7f76d6bc admin admin] PATH_INFO: `/v3/users/aa904af8376d4753b5611ebfe7653b29` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:09:14.667845 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77305b4-b94d-4c2b-ae2d-c20c7f76d6bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa904af8376d4753b5611ebfe7653b29)` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:09:14.669829 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77305b4-b94d-4c2b-ae2d-c20c7f76d6bc admin admin] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:09:14.729848 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-e77305b4-b94d-4c2b-ae2d-c20c7f76d6bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa904af8376d4753b5611ebfe7653b29'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:09:14.742687 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-e77305b4-b94d-4c2b-ae2d-c20c7f76d6bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa904af8376d4753b5611ebfe7653b29'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:09:14.754275 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-e77305b4-b94d-4c2b-ae2d-c20c7f76d6bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa904af8376d4753b5611ebfe7653b29'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:09:14.764075 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-e77305b4-b94d-4c2b-ae2d-c20c7f76d6bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa904af8376d4753b5611ebfe7653b29'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:09:14.764792 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.notifications [None req-e77305b4-b94d-4c2b-ae2d-c20c7f76d6bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa904af8376d4753b5611ebfe7653b29'} {{(pid=80459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:09:14.771200 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1430/2890] 10.222.0.26 () {58 vars in 1223 bytes} [Thu Jun 23 02:09:14 2022] DELETE /identity/v3/users/aa904af8376d4753b5611ebfe7653b29 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:09:14.950995 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d45f23-bf3e-4cdc-bc0f-cd2c206e30b4 admin admin] Authenticating user token {{(pid=80460) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:09:14.952558 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.fernet_utils [None req-72d45f23-bf3e-4cdc-bc0f-cd2c206e30b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80460) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:09:14.968545 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d45f23-bf3e-4cdc-bc0f-cd2c206e30b4 admin admin] Validating token access rules against request {{(pid=80460) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:09:15.013294 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:09:15.026892 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d31d9f9-18f6-447e-9b31-9f58f3084aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34e5aabcf8f5425f9c9f312f635ff77a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc3d5c56f68947b3b757a0c2a92187ac', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80460) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:09:15.027787 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d31d9f9-18f6-447e-9b31-9f58f3084aff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:09:15.028068 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d31d9f9-18f6-447e-9b31-9f58f3084aff admin admin] SCRIPT_NAME: `/identity` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:09:15.028342 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d31d9f9-18f6-447e-9b31-9f58f3084aff admin admin] PATH_INFO: `/v3/projects/1183d7efcd9847ef8bd9875f0d38a7f2` {{(pid=80460) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:09:15.029537 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d31d9f9-18f6-447e-9b31-9f58f3084aff admin admin] RBAC: Authorizing `identity:delete_project(project_id=1183d7efcd9847ef8bd9875f0d38a7f2)` {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:09:15.031491 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d31d9f9-18f6-447e-9b31-9f58f3084aff admin admin] RBAC: Authorization granted {{(pid=80460) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:09:15.078942 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-9d31d9f9-18f6-447e-9b31-9f58f3084aff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1183d7efcd9847ef8bd9875f0d38a7f2', 'request_id': 'req-9d31d9f9-18f6-447e-9b31-9f58f3084aff'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:09:15.085013 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-9d31d9f9-18f6-447e-9b31-9f58f3084aff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1183d7efcd9847ef8bd9875f0d38a7f2', 'request_id': 'req-9d31d9f9-18f6-447e-9b31-9f58f3084aff'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:09:15.094061 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-9d31d9f9-18f6-447e-9b31-9f58f3084aff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1183d7efcd9847ef8bd9875f0d38a7f2', 'request_id': 'req-9d31d9f9-18f6-447e-9b31-9f58f3084aff'} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:09:15.105034 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-9d31d9f9-18f6-447e-9b31-9f58f3084aff admin admin] The token cache is being invalidate because project 1183d7efcd9847ef8bd9875f0d38a7f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80460) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 02:09:15.105316 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-9d31d9f9-18f6-447e-9b31-9f58f3084aff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:09:15.105942 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG keystone.notifications [None req-9d31d9f9-18f6-447e-9b31-9f58f3084aff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80460) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 02:09:15.107541 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: [pid: 80460|app: 0|req: 1461/2891] 10.222.0.26 () {58 vars in 1232 bytes} [Thu Jun 23 02:09:14 2022] DELETE /identity/v3/projects/1183d7efcd9847ef8bd9875f0d38a7f2 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 02:09:24.767712 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=45,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:09:25.102220 dv-cinder-815582-9-iscsi devstack@keystone.service[80460]: DEBUG dbcounter [-] [80460] Writing DB stats keystone:SELECT=30,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80460) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 02:10:44.266524 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77305b4-b94d-4c2b-ae2d-c20c7f76d6bc admin admin] Authenticating user token {{(pid=80459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 02:10:44.268219 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-e77305b4-b94d-4c2b-ae2d-c20c7f76d6bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:10:44.382243 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77305b4-b94d-4c2b-ae2d-c20c7f76d6bc admin admin] Validating token access rules against request {{(pid=80459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 02:10:44.451697 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b903a5-4961-4a66-a2ae-560b78a278df service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5bf3baee1b843fa89b65e5485f6bebb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a44a5e0e3e2d4b8e9b9cec7b5b5c725c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 02:10:44.452663 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b903a5-4961-4a66-a2ae-560b78a278df service placement] REQUEST_METHOD: `GET` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 02:10:44.453117 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b903a5-4961-4a66-a2ae-560b78a278df service placement] SCRIPT_NAME: `/identity` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 02:10:44.453488 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b903a5-4961-4a66-a2ae-560b78a278df service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 02:10:44.456810 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.fernet_utils [None req-22b903a5-4961-4a66-a2ae-560b78a278df service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 02:10:44.469144 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b903a5-4961-4a66-a2ae-560b78a278df service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 02:10:44.470738 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b903a5-4961-4a66-a2ae-560b78a278df service placement] RBAC: Authorization granted {{(pid=80459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 02:10:44.481133 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: [pid: 80459|app: 0|req: 1431/2892] 10.222.0.26 () {60 vars in 1365 bytes} [Thu Jun 23 02:10:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 02:10:54.477002 dv-cinder-815582-9-iscsi devstack@keystone.service[80459]: DEBUG dbcounter [-] [80459] Writing DB stats keystone:SELECT=74 {{(pid=80459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}